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EXECUTIVE SUMMARY 



The New Horizons for Primary Schools (NHP) was implemented in 72 government schools 
in Jamaica, from 1998-2005. The program provided support to schools on the basis of 
needs identified through the preparation of a School Development Plan (also called a 
School Improvement Plan). This independent evaluation report first compares the schools 
in the NHP with a statistically matched comparison group of government schools, with 
respect to (a) support received for literacy and numeracy, (b) learning outcomes as 
measured by national tests in grades 3, 4 and 6, and (c) impact of NHP on learning comes. 
Second, the report assesses the effects of various school factors on student learning 
outcomes. Third, the report assesses the utility of the existing national tests for monitoring 
change over time. Finally, the report makes recommendations for better measuring NHP 
outcomes and for better assessing the impact of the NHP on student learning. 

Support for literacy and numeracy was greater in NHP schools as compared with a 
group of statistically matched non-NHP schools. In October and November, 2005, 
researchers visited 71 NHP and 67 statistically matched comparisons schools and carried 
out group interviews with the principal and experienced teachers . The survey asked about 
innovative mathematics and literacy programs at the school, in-service teacher training, 
governance and leadership training, parent education and training, supplementary reading 
and math materials, computer use in schools and training teachers about computers, 
training resource teachers, nutrition and health programs, integrating statistical data bases, 
and linking project schools with national EMIS. In all areas, other than parent education 
and health and nutrition, NHP schools were more advantaged than matched non-NHP 
schools. 

Learning outcomes in NHP schools and matched non-NHP schools fluctuated over 
time along with national fluctuations in test results and were not consistently higher 
in NHP schools as compared to matched non-NHP schools. National school-level 
performance data on the Grade 3 Diagnostic test and the Grade 4 Literacy test and 
individual student performance data on the Grade 6 Achievement test (GSAT) were 
obtained and analyzed. No differences were observed in seven of the ten tests analyzed. 

The performance of students in NHP schools was higher than the performance of students 
in the statistically matched non-NHP schools on two tests: 

• 2004 Grade 3 Communications Task I 

• 2004 Grade 6 Communications Task I 

The performance of students in NHP schools was lower than the performance of students 
in the statistically matched non-NHP schools on one test: 



^ Although 72 schools originally participated in NHP, one school closed. 
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• 2005 Grade 6 Communications Task I 

The performance of students in NHP schools was no different from the performance of 
students in the statistically matched non-NHP schools on the following seven tests: 

• 2004 Grade 3 Math 

• 2004 Grade 3 Language 

• 2004 Grade 4 Literacy 

• 2004 Grade 6 Math 

• 2004 Grade 6 Language 

• 2005 Grade 6 Math 

• 2005 Grade 6 Language 

Multiple regression analysis of the effects of various NHP inputs on school average 
literacy and numeracy found few systematic impacts of any inputs. However, at the 
margin: 

• Schools that were provided with more different types of supplementary reading 
materials had a smaller share of students performing at non-mastery and less-than- 
mastery levels on the Grade 4 Literacy test 

• Schools where reading resource teachers received more days of training also had a 
smaller share of students performing at non-mastery on the Grade 4 Literacy test 
and higher school averages on the GSAT2005 language test 

• Schools with a functioning computer for administrative purposes and schools with 
internet access had higher GSAT math and language scores 

At the same time, GSAT math and language scores were lower in schools receiving more 
in-service teacher training on the revised curriculum and lower in schools where principals 
participated in peer learning, with these inputs provided possibly in response to school 
needs. 

The existing national tests have drawbacks that limit their utility as USAID PMP 
indicators. Each of the national tests was analyzed to assess their utility as monitoring 
indicators. Drawbacks differ from test to test, and the principal ones are: 

• Possible changes in the timing and content of the Grade 3 Diagnostic test 

• Lack of precision in reporting sub-scores and total scores from the reporting sheets 
of the Grade 4 Literacy test, which reports only non-mastery, near-mastery or 
mastery levels of performance 

• Lack of annual, post-test equating, through empirical IRT methods, to place test 
results on a common scale to ensure comparability of measures over time; this was 
a drawback for all tests, including the GSAT. 

Monitoring and evaluation could be strengthened, to better measure the impact of the 
NHP and provide guidance for system improvement. We recommend that two steps be 
taken for better monitoring of changes over time: 

• Facilitate statistical equating of national tests through providing the MOEYC/SAU 
with appropriate technical support 
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• Encourage the MOEYC to collect actual Grade 4 Eiteracy test item responses and 
scores, not just mastery levels 

We also recommend that three steps be take for better measuring the impact of the NHP: 

• Prior to implementing the new NHP, create a prospective comparison group 
through propensity score matching (PSM) of all NHP schools with a sample of non- 
NHP schools; use the most recent data from the Annual School Census and 2005 
national test results to create the propensity scores 

• Beginning in 2006, regularly collect data from all NHP and all statistically matched 
non-NHP schools; install the JSAS in the statistically matched non-NHP schools to 
routinely collect administrative data; replace obsolete computers in NHP schools 
and provide computers as needed to matched non-NHP schools; monitor use of 
JSAS. 

• Use independent evaluators to (a) extract routine administrative data, (b) collect any 
additional data as necessary, and (c) use statistical methods to assess the impact of 
the NHP program as a whole and the contribution of various interventions to 
improvements in learning outcomes. 
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CHAPTER 1: BACKGROUND AND KEY RESEARCH 

QUESTIONS 



The New Horizons for Primary Schools Project (NHP) was initiated in school year 
1997-98 and fully rolled out in school year 1998-99. The objective of the program was 
to improve the language arts and mathematics performance of 72 of Jamaica’s poorer 
performing schools, through a school-based model of intervention. School 
improvement plans (SIPs) were to be developed for each school, with interventions 
selected from a menu of ten project interventions (see Box 1) in accordance with each 
school’s need. 



Lead institutions were 
identified for each of the NHP 
interventions, with an 
institutional contractor taking 
responsibility for all but three 
of the interventions. The 
MOEYC’s Professional 
Development Unit (PDU) was 
responsible for training of 
resource teachers and the 
MOEYC in partnership with 
the National Council on 
Education (NCE) was responsible for providing governance and leadership training for 
schools, communities and parents and for offering parent education and training. In the 
spring of 1999, a diagnostic survey of all NHP schools was undertaken, to assess the 
schools’ training and other needs (Juarez and Associates, June 1999). 



Box 1 : New Horizons Interventions 

1 . Developing innovative mathematics and literacy programs 

2. Providing in-service teacher training in reading and 
mathematics 

3. Providing governance and leadership training for schools, 
communities, parents 

4. Offering parent education and training 

5. Facilitating selective nutrition and health programs 

6. Providing reading and mathematics materials 

7. Establishing computer use in school and training teachers in 
educational teohnology 

8. Training resource teachers 

9. Integrating databases using MIS 

10. Improved school management through EMIS 



Systematic evidence regarding the extent to which various interventions were 
implemented in the NHP schools is modest, but shows improvement over time. Some 
evidence comes from evaluations of School Development Plans (SDPs), which are the 
“focal point of NHP’s approach to school governance” and also are essential in the 
needs assessment process (Dye et al, 2002, p. 11). One evaluation codified the quality 
of literacy and numeracy initiatives in SDPs for NHP schools as of November 1999, 
based on SDPs received from approximately three-quarters (56 of 72) of the NHP 
schools; of these, the quality of fewer than 20 percent was judged “satisfactory” and the 
quality of about 40 percent was judged “weak” (Juarez and Associates, December 
1999). Only about half the SDPs included a statement of actions that the school would 
take to reach their specific literacy or numeracy attainment target. The report notes that 
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“very few schools appear to be in a stated position of readiness to deal with literacy and 
numeracy in their schools” (Juarez and Associates, December 1999). This number was 
apparently higher a few years later. An analysis of 56 SDPs in 2003 judged all but four 
of them to be “good” or “very good.” (Summary Evaluation of School Improvement 
Plans (SIP) of NHP, Spring 2003). This later evaluation, however, noted that half (28 of 
56) of the NHP schools for which SIPs were available lacked the desired three-year 
action plan for implementation of the program. 

As anticipated, the NHP interventions were not implemented uniformly across all 72 
schools; implementation varied across schools in accordance with local needs. For 
example, only 14 NHP schools received breakfast programs. The intensity of the 
interventions also varied, with training program duration lasting from a few hours to 
several days. Table 1.1 summarizes the main features of the implemented program. 

Table 1.1 Features of NHP as Implemented by 2003 



Intervention 


Implementation 


1 . Developing innovative 
mathematics and literacy 
programs 


1 00s of site visits, deployment of 1 6 “NHP Associates” to work at 
classroom level 


2. Providing in-service teacher 
training in reading and 
mathematics 


Consolidated with #8 


3. Providing governance and 
leadership training for schools, 
communities, parents 


Procurement of governance and Leadership Coordinator and Officers 

Examination of SDPs 

Site visits in 60 NHP schools 

Finalized Manual on Governance and Leadership Training for School 
Boards and Principals 
NHP Principals’ Workshops 
Other training 


4. Offering parent education and 
training 


National Parenting Conference (1999, 2002) 


5. Facilitating selective nutrition 
and health programs 


Subsidy of breakfast program in 14 schools 
Community mobilization to sustain program 

Teacher training on integrating health and nutrition in teaching of core 
subjects 

Nutrition Specialist and community development specialist. 


6. Providing supplementary 
reading and mathematics 
materials 


Supplementary materials and equipment distributed to schools 


7. Establishing computer use in 
school and training teachers in 
educational technology 


5 “technology-intensive” NHP schools established 

Two three-day and one overlapping six-day Educational Technology 

Workshops held for teachers in 5 NHP schools (2002) 

One-week consultancy on use of technology for student literacy 
Consultations with 72 school principals on incorporating technology into 
SIPs 


8. Training resource teachers 


Trained 180 Math and Literacy Resource Teachers in workshops and 
in-school training activities 


9. Integrating databases using MIS 


JSAS software 5.0 developed and utilized within NHP schools 
Support guides and training manuals for 200 non-NHP schools 
prepared 


10. Linking Project Schools to 
EM IS Network 


25 large and medium schools received additional computers (2002) and 
1 40 computers were networked 



Source: O’Neil, October 2003 
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Over the period of implementation of the NHP a large number of formative and other 
evaluations were carried out; nearly 100 were catalogued by the Curriculum and 
Support Services Unit of the MOEYC (O’Neil 2003). However, none of these studies 
addressed, in a comprehensive manner, a series of questions posed by USAID: 

1. Have NHP schools made achievement gains over the years under review? 

2. What factors in the project schools may have affected gains or lack of gains? 

3. Is the use of mastery /non-mastery /non-mastery categories on the GSAT masking 
real gains in student achievement in schools? 

4. Is the GSAT the best measure of student performance for the project schools, 
considering that it is based on the content delivery system of the Old Curriculum? 

5. How can valid measures of students’ computational skills in numeracy be assessed 
for students who are unable to comprehend the language in which most numeracy 
items are couched in the GSAT examination? 

6. How has “social promotion” to grade 6 affected average performance results among 
students 

7. How effective were the indicators used for tracking the results of the NHP and what 
suggestions could be made for the future? 

8. How effective are the methodologies used to collect data? 

In 2005, the Academy for Educational Development carried out an evaluation of the 
NHP that addressed many of these questions. However, at the time of the investigation 
there were several shortcomings with regard to available data. Questions were raised 
about the adequacy of the GSAT to measure changes in literacy and numeracy of low 
performing students. Eittle was known about the inputs received by comparable non- 
NHP schools, leaving open the possibility that non-NHP schools had benefited from 
resources that would have been comparable to those received by schools in the NHP 
program. This, in turn, could have offset anticipated achievement differences between 
NHP and matched non-NHP schools. The present report addresses these questions. 
Chapter 2 describes the process of identifying matched non-NHP schools. Chapter 3 
reports the findings of the school survey that collected data from the NHP schools and a 
set of matched non-NHP schools; it describes the support provided to NHP schools and 
compares this with support provided to other, matched, non-NHP schools. Chapter 4 
extends the analysis presented in Eockheed, Harris, Gammill and Barrow (2004) to 
examine the effects of NHP on student literacy and numeracy in Grades 3 and 4. 
Chapter 5 deepens the analysis presented in Eockheed et al (2004) by employing 
hierarchical linear modeling to assess the impact of NHP on student performance on 
Grade 6 tests and Chapter 6 explores the question of what factors contribute to 
increased learning. Chapter 7 analyzes four national tests (Grade 1 Assessment, Grade 
3 Diagnostic Grade 4 Eiteracy and Grade 6 Achievement) and assesses their utility in 
monitoring change over time and as PMP indicators. Chapter 8 presents our 
conclusions and recommendations. 

Eor this evaluation, the MOEYC provided us access to Grade 3 and Grade 4 data from 
the school year 2003-2004, to GSAT data from 2005 (in addition to GSAT data 1999- 
2004 provided previously), and to selected data from the Annual School Census of 
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2004 (in addition to Annual School Census data 1999-2003 provided previously). In 
addition, we fielded a survey of the 71 extant NHP schools and a statistically matched 
comparison group of non-NHP schools to obtain independent evidence regarding the 
degree to which NHP support had reached the schools and whether non-NHP schools 
had received comparable support, although from other sources. 
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CHAPTER 2. MATCHED COMPARISON SCHOOL GROUPS 

We used propensity score matching (PSM) to identify a set of government schools in 
Jamaica that were matched with NHP schools in 1999 across a wide range of 
characteristics, but that did not participate in the program. We identified three different 
comparison groups, using three different PSM techniques: (a) PSM with replacement, (b) 
PSM with non-replacement, and (c) PSM with replacement and substitution. Propensity 
score matching and the three different comparison groups are described in Annexes A and 
B, respectively. The results do not differ substantially across the three groups. 

Criteria for Matching NHP Schools with Non-program Schools 

Of the 72 NHP schools, we were able to match 71 with equivalent schools not participating 
in the program (Annex B). The number of unique matched schools was 58 for PSM with 
replacement (i.e., some NHP schools were matched with the same non-NHP schools), 71 
for PSM without replacement and 66 with PSM with replacement and substitution (i.e., 
reasonable substitutions were identified for selected non-NHP schools). 

To the extent possible, using 1999 school census data available for all government schools 
in Jamaica, we matched schools on the basis of eight criteria that were initially used to 
place schools in the program; these were: (a) performing at or below the national mean in 
language arts and mathematics in the National Assessment Programme, (b) performing at 
or below the national mean in language arts, mathematics, science and social studies in the 
National Assessment Programme, (c) geographic location, (d) evidence of Board, or 
principal and teachers taking action to address the under- achievement of students in the 
school, (e) active functioning School Board or SCOPE Committee, (f) recipient of grant for 
Jamaica School Investment fund or civil works in the IDB PIEP project, (g) potential for 
providing inspiration and leadership in the project, (h) participation in other initiatives 
complementary to the project. In 1998, 194 schools met these criteria (PIU, December 
1998) and were eligible for selection; data related to some of these criteria were included in 
the 1998 or 1999 School Census for all schools. 

In addition, we included regional “dummy” variables and indicators for four other 
characteristics of schools and teachers that we hypothesized were important determinants 
of student achievement and for which we could obtain data from the School Census: (a) 
teacher quality, (b) teacher experience, (c) poverty level of school community, and (d) size 
of school. These twelve characteristics and the data sources for each are summarized in 
table 2. 1 . 
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Table 2.1. Criteria for school selection into New Horizons for Primary Schools program 



NHP Selection Criteria 


Indicator from School Census or G SAT 


Performing at or below the national mean in language 
arts and mathematics in the National Assessment 
Programme, 


School mean GSAT Scores on language arts and 
mathematics, school year 1998-99 


Performing at or below the national mean in language 
arts, mathematics, science and social studies in the 
National Assessment Programme, 


School mean GSAT Scores on language arts, 
mathematics, science and social studies, school year 
1998-99 


Geographic location 


School Census classification of school as rural, remote 
rural 


Evidence of Board, or principal and teachers taking action 
to address the under-achievement of students in the 
school. 


None 


Active functioning School Board or SCOPE Committee, 


School Census: presence of School Board or SCOPE 
Committee 


Recipient of grant for Jamaica School Investment fund or 
civil works in the IDB PIEP project, 


None 


Potential for providing inspiration and leadership in the 
project, 


None 


Participating in other initiatives complementary to the 
project. 


None 


Other school factors 




Teacher quality in Grades 1-6 


School Census: Percent teachers with CXC as highest 
level of school attainment 




School Census: Percent master teachers in school 


Teacher experience in Grades 1-6 


School Census: Average number of years experience 
as a teacher 




School Census: Average number of years experience 
in the school 




School Census: Percent teachers with less than two 
years experience 


Poverty level of school community 


School Census: School breakfast program 


Size of school 


School Census: Number of teachers. Grades 1 -6 




School Census: Number of students. Grades 1 -6 




School Census: School on Shift 


PTA 


School Census: presence of PTA 



Comparison of NHP schools with Matched Schools in 1999 

All three groups of non-NHP schools were well-matched with the NHP schools at the 
outset of the program. Inspection of school, teacher and student characteristics of the NHP 
and matched non-NHP schools as of 1999 demonstrates the similarity of the four sets of 
schools (table 2.2). For none of the initial 1999 characteristics, including average student 
performance on the GSAT1999, are there statistically significant differences between the 
NHP schools and any of the three comparison groups, other than for inclusion on the list of 
initially qualified schools. NHP schools do appear to be larger, in terms of both numbers of 
teachers and numbers of students, but these differences are not statistically significant. 
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Table 2.2: School and teacher characteristics, NHP schools and PSM comparison groups, 1998-1999 



Number of Observations 


NHP 

Schools in 
1999 


Comparison Schools 




PSM with 
replacement 


PSM with non- 
replacement 


PSM with 
substitution 


72 


58 


71 


66 


School Characteristics 1999 










Size: Enrollment in Grade 1-6 


382 


298 


299 


298 






(-1.2373) 


(-1.3185) 


(-1 .2843) 


Size: Number of teachers in Grades T6 


12.89 


10.38 


10.63 


10.25 






(-1.2986) 


(-1.2491) 


(-1 .4062) 


School shift (percent) 


13 


10 


9 


9 






(-.3796) 


(-0.7576) 


(-.6693) 


Rural location (percent) 


39 


50 


44 


46 






(1 .2672) 


(.6488) 


(0.8757) 


Remote rural location (percent) 


24 


26 


27 


27 






(0.294) 


(.4806) 


(0.4388) 


Breakfast program (percent) 


18 


14 


17 


15 






(-.6524) 


(-.1418) 


(-.4931) 


Active PTA (percent) 


97 


97 


97 


96 






(-.2184) 


(-.0284) 


(-.5344) 


Board and/or scope (percent) 


88 


90 


91 


87 






(.3796) 


(0.7576) 


(-.1625) 


On list of initially qualified schools (percent) 


100 


14 


19 


69 






(-21.0494) 


(-17.6422) 


(-5.6918) 


Teachers Characteristics 1999 










Qualifications: CXC highest (percent) 


27 


31 


31 


30 






(1.0321) 


(1.0759) 


(.7593) 


Qualifications: Certificate highest (percent) 


67 


65 


65 


65 






(-.6455) 


(-.7593) 


(-0.6342) 


Qualifications: Bachelors degree (percent) 


5 


4 


4 


5 






(-.9632) 


(-.7250) 


(-.2038) 


Qualifications: Masters degree (percent) 


39 


43 


39 


41 






(.3019) 


(0.0071) 


(.1243) 


Experience: Mean years in service at grade 1 to 6 


15.25 


14.93 


14.79 


14.55 






(-.2529) 


(-.3959) 


(-.7505) 


Experience: Mean years in service at grade 1 to 6 in 


11.05 


9.94 


9.78 


9.9 


school 




(-1.1484) 


(-1 .4062) 


(-1.2611) 


Experience: Percent of teachers in school with 2 or 


24 


21 


20 


22 


less years experience 




(-.7613) 


(-.9924) 


(-.6189) 


Student Achievement (GSAT school means) 1999 










Mathematics 1999 


28.34 


29 


28.65 


27.63 






(.8233) 


(0.3932) 


(-.9149) 


Science 1 999 


22.85 


23.36 


23.05 


22.42 






(.7687) 


(.3198) 


(-.688) 


Social Studies 1999 


33.84 


34.48 


33.93 


33.11 






(.678) 


(0.0932) 


(-.8349) 


Language 1999 


32.87 


33.59 


33.14 


32.34 






(.7037) 


(.2807) 


(-.53) 


Communications task 1 1999 


2.36 


2.36 


2.33 


2.32 






(0.0657) 


(-.3043) 


(-.5746) 


Communications task 2 1999 


1.33 


1.28 


1.27 


1.26 






(-.6758) 


(-.9488) 


(-1.1981) 



Note: In parentheses is the t-value of the difference between the group mean and the NHP school mean. 



The remaining chapters in this report utilize these three comparison groups for assessing 
the effects of the NHP. In addition, schools from the PSM with substitution group were 
surveyed along with the NHP schools to assess the degree of support received by schools. 
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CHAPTER 3. DID NHP SCHOOLS RECEIVE MORE 
SUPPORT THAN NON-NHP SCHOOLS? 



Given that the NHP was implemented over a period of time - 1999 through 2005 - when 
other programs were being implemented in primary schools in Jamaica, a key question was 
whether the schools in the NHP program received more support that other, comparable 
schools, who might have been receiving assistance through other programs; for example, 
the PESP provided support to 15 “IT Pilot” schools and 12 “Lighthouse” schools. In 
October and November, 2005, researchers visited 71 NHP and 67 statistically matched 
comparison (PSM with substitution) schools and carried out group interviews with the 
principal and experienced teachers. The survey asked about innovative mathematics and 
literacy programs at the school, in-service teacher training, governance and leadership 
training, parent education and training, supplementary reading and math materials, 
computer use in schools and training teachers about computers, training resource teachers, 
nutrition and health programs, integrating statistical data bases, and linking project schools 
with national EMIS. The survey instrument and summary of responses are provided in 
Annex C. 



In most areas, NHP schools were more advantaged than matched non-NHP schools. Table 
3.1 summarizes the results from this survey according to the ten interventions of the NHP 
as initially conceived. The results show that NHP schools were substantially more 
advantaged than the matched non-NHP schools, in all but parent education and health and 
nutrition. In the remainder of this chapter, we discuss differences between NHP and 
matched non-NHP schools with respect to the areas in which the NHP schools were more 
advantaged, aggregated into: (a) support for the Revised Primary Curriculum (RPC), (b) 
governance and leadership training, (c) in-service teacher training, and (d) computers and 
IT support. We then discuss the two areas in which NHP and matched non-NHP were 
similar. 



Table 3.1: Differences between NHP and Matched non-NHP schools in 10 NHP intervention areas 



NHP intervention 


NHP vs. Matched non-NHP schools 


1. innovative mathematics and literacy programs 

2. in-service teacher training 

3. governance and leadership training 

4. parent education and training 

5. selective nutrition and health programs 

6. supplementary reading and math materials 

7. establishing computer use in school and training teachers 

8. training resource teachers 

9. integrating databases (JSAS) 

10. linking project schools with EMIS 


NHP more advantaged 
NHP more advantaged 
NHP more advantaged 
No difference (both low) 

No difference 

NHP more advantaged (both high) 

NHP more advantaged 

NHP more advantaged 

NHP more advantaged 

NHP more advantaged (both low) 



15 




Lockheed, Harris, Gammill, Barrow and Jayasundera 



NHP Inputs, Outcomes and Impact 



NHP Schools Generally More Advantaged 

NHP schools reported significantly more support for reading and literacy at the primary 
level, including preparing School Improvement Plans, and receiving supplementary reading 
and math materials, governance and leadership training, in-service teacher training, and 
computers and IT support. 

Support for mathematics and literacy programs 

A major objective of the NHP was to help school develop innovative mathematics and 
literacy programs that would be effective in boosting the performance of lower performing 
students. However, NHP schools were not the only schools in Jamaica receiving this 
support from the MOEYC and other donor programs. Some of these initiatives were known 
variously as professional development centers, best practice demonstration schools, 
learning centers, lighthouse schools, and pilot schools for the Revised Primary Curriculum 
or for Instructional Technology. In order to determine if the matched non-NHP schools 
were benefiting from these types of programs, we asked all schools how other schools 
might have heard about the school. A higher share of NHP schools (86 percent) than 
matched non-NHP schools (51 percent) reported that they might be recognized by others, 
and this difference was statistically significant (p < .001). Many NHP schools mentioned 
that they were NHP schools, but 72 percent of NHP schools mentioned another reason they 
might be recognized by others. 

More NHP schools than matched non-NHP schools reported having school improvement 
plans; however, concurrently with implementation of the NHP, the MOEYC was 
implementing other activities to support SIPs and SDPs. As a consequence, nearly all 
schools — 94 percent of NHP and 85 percent of matched non-NHP schools — reported 
having a School Improvement Plan (SIP) for the purpose of improving reading and math 
skills of students in the school; this difference was, however, statistically significant ( p < 
.05). A higher share of NHP schools (84 percent) than matched non-NHP schools (64 
percent) reported having specific activities to support the Revised Primary Curriculum (p < 
.01). However, there were no differences between NHP and matched non-NHP schools in 
the presence of such specific activities as a Drop Everything and Read program, the 
presence of Competence Shelter (most schools did not have one) or being a Summer 
Eiteracy Camp venue (most schools reported being one). 

A higher share of NHP schools reported receiving supplemental reading and math 
materials, and NHP schools reported receiving a greater variety of materials, including 
books and computer software, all in support of the RPC (figure 3.1). Ninety-three percent 
of NHP and 79 percent of matched non-NHP schools received supplementary reading 
materials (p < .05), and 85 percent of NHP and 54 percent of matched non-NHP schools 
received supplementary math materials (p < .001). About half of both types of schools 
reported receiving books for student reading (44 percent of NHP and 45 percent of matched 
non-NHP), but NHP schools received a greater variety of types of books including story 
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books, shared reading books, and books for student independent reading (p < .10). In 
support of math, NHP schools received more types of math materials than matched non- 
NHP schools (p<.001). For example, 51 percent of NHP and 1 percent of matched non- 
NHP schools received calculators for students, 72 percent of NHP and 27 percent of 
matched non-NHP schools received manipulative such as blocks; 57 percent of NHP and 
34 percent of matched non-NHP schools received math activity books; 

Approximately 80 percent of both types of schools reported having a local sponsor (such as 
Kiwanis, Rotary, Lion’s Club, a local church. Adopt- a-School) that provided cash or in- 
kind resources to the school (82 percent of NHP and 76 percent of matched-non-NHP 
schools). Ninety-six percent of NHP schools and 79 percent of matched non-NHP reported 
that these resources were for the purpose of improving students’ reading and math skills (p 
< .01), and about the same share reported that the resources directly supported a SIP (93 
percent of NHP and 73 percent of matched non-NHP (p < .001) 

Figure 3.1 School Improvement Plans, Support to Revised Primary Connecticut, Supplemental 
Materials for Reading and Math (% of schools) 




-| — I — r 

SIP or SDP Activities to Reading Math Materiais 

support RPC Materiais 



□ NHP 

□ non-NHP 



Providing governance and leadership training for schools, communities and parents 

A core feature of the NHP was its emphasis on promoting school improvement through 
governance and leadership training for school principals, school boards and the community. 
NHP schools were advantaged in training for leadership (figure 3.2). The training was 
designed to support school improvement and school development action plans SIPs and 
SDPs). Twice as many NHP schools (54 percent) reported that they had a trained School 
Improvement Action group, compared with 26 percent of matched non-NHP schools 
having such groups, a statistically significant difference (p<.01). 
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Figure 3.2. Leadership Training in NHP and matched non-NHP schools for Principals, Community 
Coordinators and School Boards 
(% of schools reporting) 








□ NHP 

□ non-NHP 











Principal training Comm. Coord. Board Workshops 



Interestingly, about the same share of NHP principals and principals of matched non-NHP 
schools, over half of both, reported having received training (e.g. 58 percent of NHP and 59 
percent of matched non-NHP school principals participated in training provided by Mt. St. 
Vincent University; 55 percent NHP and 58 percent of comparison principals received 
training in implementing the RPC). However, the variety of training received by NHP 
principals was greater than that received by principals of matched non-NHP schools (p < 
.01). In addition many more NHP schools recruited and trained communication 
coordinators compared with matched non-NHP schools (86 percent compared with 53 
percent, p < .001) and NHP schools were also more likely to receive School Board 
workshops delivered by the National Council on Education. 



Providing in-service teacher training in reading and math 

An innovative aspect of the NHP was the substantial training provided to “support 
teachers” at the school level. The survey describes support teachers, as follows: “In some 
primary schools, experienced teachers have been identified as persons who can provide 
support to other teachers. These support teachers are called Resource Teachers or 
Literacy/Numeracy/Assessment Coordinators. Some larger schools have Support Teachers 
that help other teachers on a grade by grade basis, called Grade Coordinators. . . . We call 
all of these types of teachers Support Teachers. ” The objective of asking the question in 
this way was to elicit information about the type of support provided in-school by teacher 
peers. 
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A higher share of NHP schools reported having support teachers, and the support teachers 
in NHP schools received more types of training to help them work with their peers (table 
3.2). On average, support teachers in NHP schools received twice as much training as 
support teachers in the matched non-NHP schools: six days of training during the school 
year 2004-2005, compared with less than three days of training received by support 
teachers in matched non-NHP schools (p < .05). 



Table 3.2. Support Teachers in NHP and matched non-NHP Schools 





NHP Schools (%) 


Matched Non-NHP Schools 
(%) 


Stat. Sig. (p < ) 


Presence of support teachers 


Any support teachers 


86 


67 


.01 


Math support teachers 


88 


44 


.001 


Literacy support teachers 


91 


47 


.001 


Training of support teachers 


How to teach other teachers 


50 


19 


.001 


Cooperative learning 


82 


36 


.001 


Peer to peer observation 


40 


10 


.001 



The survey asked about a wide range of opportunities for teachers to receive in-service 
training; respondents from NHP schools identified significantly more training and more 
different types of training than did respondents from the matched non-NHP schools. Ninety 
percent of NHP schools reported that in the past five years, teachers of grades 1-6 
participated in school-based in-service teacher training provided by outside specialists, 
whereas only 60 percent of matched non-NHP schools reported such training. Figure 3.3 
shows the estimated percent of teachers who received more than ten days of training on the 
RPC since it was introduced; NHP schools reported a significantly higher share of teachers 
having received training. In addition NHP schools report a higher amount of training by 
outside experts, such as provided by the NHP. The differences are statistically significant at 
the p < .001 level. 



Figure 3.3 In-service Teacher Training Received by NHP and matched non-NHP schools (% of schools) 
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In addition, NHP schools reported receiving a wider variety of types of in-service teacher 
training for both literacy and math, including clinical supervisory practice, demonstrations 
by specialists, training in cooperative learning and unannounced observations by Ministry 
officials (figure 3.4 for literacy). The differences were statistically significant( p < .001). 

Figure 3.4. In-service Training to Improve Literacy (% of schools reporting) 
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NHP teachers also benefited from on-site training provided by in-school resource teachers 
in math and literacy. About twice as many NHP schools had resource teachers for math and 
literacy (88 percent compared with 44 percent in matched non-NHP schools). Moreover, 
the content of in-service training provided by in- school resource teachers covered more 
topics of math and literacy in NHP schools than in matched non-NHP schools. For example 
7 1 percent of NHP schools reported training in “making mathematics fun” compared with 
25 percent of matched non-NHP schools reporting training on that topic. In general, about 
twice as many NHP as matched non-NHP schools reported training on each topic. 

Computers for teaching and administration 

According to NHP program documents, the final two years of the program placed increased 
emphasis on providing computers to NHP schools. The survey shows the effects of this 
effort: 72 percent of NHP and 28 percent of matched non-NHP schools had working 
computers for students (p < .001); 79 percent of NHP and 38 percent of matched non-NHP 
schools had a computer resource teacher (p < .001); and 21 percent of NHP schools and 7 
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percent of matched non-NHP schools received software as supplemental materials for 
literacy and math. 

Figure 3.4. Computers for Teaching and Administration 
(% of schools) 




In addition, a substantially higher share of NHP schools had computers that could be used 
for administrative purposes. For example, 87 percent of NHP and 67 percent of matched 
non-NHP schools had a computer for school records (p < .05) and virtually all NHP 
schools had the ISAS software installed. In support of the objective to link project schools 
with a central EMIS, 41 percent of NHP compared with 12 percent of matched non-NHP 
schools had internet access (p < .001); a remaining large share of NHP schools, however, 
does not benefit from access to the internet. 

NHP Schools No Different in Two Areas 

In two areas, parent education and training, and health and nutrition programs, we found no 
difference between NHP schools and matched non-NHP schools. In the latter case, this is 
undoubtedly due to the relatively small number of 14 NHP schools targeted for such 
programs. In the case of parent education and training, the survey found inconsistent 
responses. 

Parent education and training 

Neither NHP nor the matched non-NHP schools reported substantial programs of parent 
education and training, but responses to open- and closed-ended questions were 
inconsistent. On an open-ended question, more than half (53 percent) of NHP and 70 
percent of matched non-NHP schools reported that training was provided for community 
and parents, with about one-third of schools reported offering some type of “parenting” 
workshops for parents (38 percent of NHP schools and 33 percent of matched non-NHP 
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schools). However, in response to direct questions about programs for parents {“Does this 
school have afternoon or evening programs for parents to help them understand the 
Revised Primary Curriculum so they can help their children?”), only a small fraction of 
either NHP or matched non-NHP schools reported having such programs: nine percent of 
NHP and 17 percent of matched non-NHP schools. Only about ten percent of both types of 
schools offered a Family Literacy program (such as Reading Starts with Us). 

Selective nutrition and health programs 

The survey asked about the identification of and programs for “special needs” students. 
Few schools reported professional screening of children by a doctor, nurse or psychologist. 
Instead, 86 percent of NHP and 93 percent of matched non-NHP schools reported that 
screening was carried out by classroom teachers. Growth monitoring was reported more by 
NHP schools (28 percent) than by matched non-NHP schools (14 percent). 

Programs for “special needs” students were typically school meals programs, with 35 
percent of NHP schools and 21 percent of matched non-NHP schools reporting that 
“special needs” students participated in breakfast programs. In all 64 percent of NHP and 
60 percent of non-NHP schools reported some form of meal program for “special needs” 
students. About 15 percent of both types of schools participated in the PATH program. 

Over 90 percent of both types of schools reported that a cooked meal was available at the 
school and abut half of both types of schools reported availability of nutribuns and a drink. 
NHP schools reported serving meals slightly more regularly than matched non-NHP 
schools but over 85 percent reported serving meals on a daily basis. Ninety-seven percent 
of all schools provided free meals to some students. 
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CHAPTER 4. DID THE NHP BOOST LITERACY AND 
NUMERACY IN GRADES 3 AND 4? 



This chapter examines the effects of the New Horizons for Primary Education Program 
(NHP) on the literacy and numeracy of Grade 3 and Grade 4 students in Jamaica. It 
concludes that the performance of students in NHP schools is no higher than the 
performance of students in a statistically matched comparison group of non-NHP schools 
on either the Grade 3 Diagnostic test or the Grade 4 Literacy test. 

Background on the NHP Relative to Grades 3 and 4 



The NHP was initiated in 1999 with an objective of improving the literacy and numeracy of 
primary school students attending some of the poorest schools in Jamaica. Its introduction 
coincided with the introduction of the Revised Primary Curriculum (RPC) in NHP schools 
in all grades in the school year 1999-2000 and with the phased introduction of the RPC in 
all primary schools beginning with Grades 1 and 4 in the school year 2000-2001. By the 
school year 2003-4, therefore, students in Grades 3 and 4 in all government primary 
schools, including NHP schools, would have experienced the RPC since the beginning of 
Grade 1. 

Primary teachers in all government schools received training about teaching the RPC, but 
teachers in NHP schools received additional support, as did the schools themselves. This 
support included both training and provision of school inputs, as summarized in table 1. 

Did this additional support boost the literacy and numeracy of Grade 3 and 4 students in 
NHP schools, as compared with students in comparable non-NHP schools? This chapter 
addresses this question. 



Background oe Follow-up Analyses 



In an earlier report (Lockheed et al. 2004), AED assessed the impact of the NHP on the 
literacy and numeracy of Grade 6 students, using the 2003-4 GSAT tests as indicators. This 
approach had three main shortcomings. Lirst, students completing Grade 6 in the school 
year 2003-4 had not experienced the RPC throughout primary school, and therefore their 
teachers may not have participated in the training provided by the NHP, even though their 
schools may have benefited more broadly from the program. Second, the GSAT is a “high 
stakes” test, which is designed to identify the highest performers; since the NHP is targeted 
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at raising levels of basic literacy and numeracy, the GSAT is not likely to be sensitive to 
changes at the lower ends of the performance scale. Third, the 2003-4 GSAT did not reflect 
the curriculum changes of the RPC, so that even if students had been exposed to the revised 
material the test would not have measured that exposure. Nevertheless, the GSAT was the 
most widely implemented and nationally reported test of student achievement at the 
primary level, and was therefore the only indicator available during the life of the project. 

To address these shortcomings of the previous analysis, it was agreed to examine the 
effects of the NHP on achievement at Grades 3 and 4, for students whose entire primary 
school experienced could have benefited from both the RPC and the NHP. For this 
analysis, 2003-4 results from the Grade 3 Diagnostic Test and the Grade 4 Literacy Test 
were obtained and analyzed. While not all schools forwarded their results on these tests to 
the central administration, results were available for more than 85 percent of all schools, 
including a similar share of NHP schools. 



Differences in Mean Achievement 

We compare NHP schools with non-NHP schools in two different ways. First, we examine 
the mean scores of schools in 2004, using t- tests. Second, we use a simple OLS regression 
of school means, where the dependent variables are the school mean scores for the Grade 3 
Diagnostic Tests and the average percent of students scoring at non- and near-mastery on 
the Grade 4 Literacy Tests in 2004, and the NHP program is considered an independent 
variable, with the school mean GSAT1999 score in math or reading as a control. 

In the earlier analysis, we used a matched-pair comparison t-test, which is a more powerful 
test of differences than the t-test for independent samples. In the present analysis, we use 
the t-test for independent samples, since PSM with replacement means that the same school 
can serve as a match for more than one NHP school, and therefore it would be counted 
multiple times when paired on a one-to-one basis with an NHP school The t-tests for 
independent samples, therefore, is based on different number of cases, depending on which 
comparison group is used. In all cases, all NHP schools with data are compared to the full 
number of unique non-NHP schools with data. The number of unique schools in each 
comparison groups is: 58 for PSM with replacement, 70 for PSM without replacement and 
66 for PSM with replacement and substitutions. 

No Differences between NHP and Non-NHP Schools at Grade 3 or Grade 4 

We found no differences in the average performance of students in NHP schools, as 
compared with any of the three comparison groups on either test. That is, neither Grade 3 
Diagnostic Test scores in math and reading nor Grade 4 Literacy Test scores were higher in 
NHP schools than in comparable non-NHP schools, as of the school year 2003-4. 
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Grade 3 Diagnostic Test 

The Grade 3 Diagnostic Test provides teachers with information about student skills in 
reading and mathematics and is intended to identify children in need of instructional 
remediation. It comprises tests of reading, math and communications. Reading and math 
tests have five subtests each (phonics, structure, vocabulary, study skills, reading 
comprehension for reading and number, measurement, geometry, algebra and statistics for 
math) with multiple-choice questions, while the communications task is graded on a five- 
point “mastery” scale (0-4). Grade 3 students are tested by their teachers towards the end 
of the school year, in the spring, with tests scored on site and summary sheets of results 
forwarded to the SAU/MOEYC. Approximately 85 percent of primary schools forwarded 
school results to the SAU/MOEYC in 2003-4, and 95 percent of Grade 3 students in NHP 
and matched non-NHP schools that forwarded scores were present for the Grade 3 tests. 

Eor only one of the comparison groups (PSM with replacement and substitution) was there 
a statistically significant difference in favour of the NHP schools, and that for one test only 
(Communications Task). Eor all other comparisons, there were no differences in 
performance on the Grade 3 Diagnostic Test between students in NHP schools and students 
in the matched non-NHP schools (Table 4.1). 



Table 4.1: Average Grade 3 Diagnostic Test Scores, NHP schools and PSM comparison groups, 2003-4 



Test Result 


NHP 




Matched non-NHP schools 


PSM with 
Replacement 


PSM with Non- 
Replacement 


PSM with 
Replacement and 
Substitution 


Average Math Score 


31.58 


32.02 


32.09 


31.03 


Average Language Score 


34.11 


34.22 


34.34 


33.42 


Average Communications Task 


1.63 


1.54 


1.57 


1.40* 


N 


59/60/60 


48/51/51 


58/62/62 


55/55/55 



* p< .05 



Grade 4 Literacy Test 

The Grade 4 Eiteracy Test is intended to identify children who are at risk of not being 
literate by the end of Grade 6. The test comprises three sections: word recognition (40 
multiple-choice items), reading comprehension (30 multiple-choice items), and 
communication/writing (scores range from 0-8). All children sit the exam in the spring of 
grade 4 (pretest), their results are scored locally, and their performance classified as 
demonstrating mastery (mastery on all three sub tests), partial mastery (mastery on 1 or 2 
subtests) or non-mastery. Children classified as having non- or partial-mastery may be 
recommended for Summer Eiteracy Camp or for repetition of Grade 4. While results are 
scored locally and forwarded electronically to the MOEYC, only the summary mastery 
classifications are provided (that is, the raw scores or sub-scores are not submitted). 
Approximately 85 percent of primary schools forwarded school results to the 
SAU/MOEYC in 2003-4, and nearly 95 percent of Grade 4 students in NHP and matched 
non-NHP schools that forwarded scores were present for the Grade 4 tests. Slightly fewer 
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students in NHP schools were present for the literacy test (93 percent) as compared with 
students from the matched non-NHP schools (95 percent) and this difference was 
statistically significant for the PSM with Replacement and the PSM with Non-replacement 
groups. 



Table 4.2: Average Grade 4 Literacy Test Scores, NHP schools and PSM comparison groups, 2003-4 



Test Result 
(Percent students) 


NHP 




Matched non-NHP schools 


PSM with 
Replacement 


PSM with Non- 
Replacement 


PSM with 
Replacement and 
Substitution 


Percent Partial Mastery 


31.91 


31.93 


32.28 


34.29 


Percent Non-Mastery 


17.84 


16.82 


16.82 


16.96 


Percent for Summer Camp 


44.32 


43.27 


43.53 


43.32 


N 


69 


57 


69 


66 



There were no differences in average performance on the Grade 3 Diagnostic Test between 
students in NHP schools and students any of the three comparison group schools (Table 
4.2). Nor were there differences in the share of students recommended for Summer 
Literacy Camp, with about 44 percent of students recommended in all groups. 



OLS Regression Results 

We next examined the effects of NHP on the Grade 3 and Grade 4 tests, using OLS 
regressions, in which the initial performance of the school, as measured by the GSAT1999, 
was held constant. We carried out this analysis for all three comparison groups and found 
no differences with respect to the impact of the NHP. Table 4.3 reports the results from the 
PSM with replacement. For none of the five regressions is there an effect of the NHP on 
average achievement at the school level. That is, other things equal, NHP and non-NHP 
schools did not perform differently in terms of the average literacy or numeracy of students 
in Grades 3 and 4, as measured by the available tests. 



Table 4.3: NHP Program Effects on School Mean Grade 3 Diagnostic Test and Grade 4 Literacy 
Mastery Scores in 2004, OLS regressions 





Grade 3 Diagnostic Test Scores 


Grade 4 Literacy Test 


Math 


Reading 


Communi- 

cations 


% Near 
Mastery 


% Non-mastery 


School GSAT1999 Math 


-.1057 










School GSAT1999 Reading 




-.0482 




-.0020 


-.0049* 


School GSAT1999 Commi 






.1939 


.0395 


.0087 


NHP school 


-.5298 


-.1576 


.0798 


-.0003 


.0068 


Constant 


35.114*** 


35.839*** 


1 .086*** 


.2913*** 


.3109*** 


N 


107 


111 


111 


126 


126 


r2 


.0077 


.0025 


.0250 


.0127 


.0454 


F 


.4034 


.1364 


1.383 


.5251 


1 .9323 



legend: * p<0.05; ** p<0.01 ; *** p<0.001 



Why should this be the case? The average characteristics of the NHP and non-NHP schools 
are very similar, and the empirical values of these characteristics as of 1999 at the outset of 
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the program did not differ statistically. The NHP supported the schools in the program with 
additional training and physical inputs not available to other similar schools. Given these 
differences in advantage documented in Chapter 3, it is difficult to understand the absence 
of differences between the NHP and matched non-NHP schools in student achievement, as 
measured by Grade 3 and Grade 4 tests. 
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CHAPTER 5: HIERARCHICAL LINEAR MODEL 
ANALYSES OF NHP EFFECTS ON GRADE 6 
ACHIEVEMENT 



This chapter examines the effects of the New Horizons for Primary Schools (NHP) 
program on student performance as measured by the GSAT. The approach compares the 
performance of GSAT test-takers from NHP schools with the performance of GSAT test- 
takers from matched, non-NHP schools, using a statistical technique (hierarchical linear 
modeling) that takes into account the nested data structure of students within schools and 
the variations in numbers of students across schools, which our previous analysis did not 
(Lockheed et al 2005). On the basis of the analysis detailed below, we conclude that the 
NHP did not boost achievement as measured by the GSAT, although it may have improved 
the performance of older test-takers as compared with younger test-takers. We speculate 
that sharp increases in enrollments in NHP schools may account for the absence of strong 
program effects. 



Analytic Samples 



The present analysis is based on the GSAT test-takers in NHP schools and two different 
comparison groups, the first estimated through propensity score matching (PSM, without 
replacement) and the second constructed on the basis of propensity scores but including a 
large number of schools initially identified as qualified for the NHP but not selected (PSM 
with substitution).^ PSM was carried out on the basis of baseline 1999 data available for all 
schools. The analysis compares 71 NHP schools"^ with each of the two comparison groups, 
for two consecutive school years: 2004 and 2005. Students were all those reporting Grade 
Six Achievement Test (GSAT) scores, for whom gender and age information were also 
available; 10 students^ in 2004 and 69 students in 2005 lacked age data. A summary of the 
analytic sample is provided in table 5.1. 



^ We also examined the PSM with replacement group; results were no different and are not reported. Details 
on the methods for identifying these comparison groups are provided in Annexes A and B.. 

Although 72 schools originally participated in NHP, one school closed. 

^ Nine for the PSM with substitution comparison group 
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Table 5.1: Analytic samples of students and schools 





NHP and PSM without replacement 


NHP and PSM with substitution 




2004 


2005 


2004 


2005 


Students (level 1) 


7289 


7845 


7081 


7645 


Schools (level 2) 


142 


141 


138 


137 



Schools were not randomly assigned to participate in the NHP. Rather, several criteria were 
used to select schools into the program, as detailed in Lockheed et al (2005). Our earlier 
analysis (Lockheed and Jayasundera 2005) found that empirical measures of the criteria 
used for selecting schools into the program were generally unrelated to the statistical 
probability of being in the program. Thus, other non-specified and unobserved criteria are 
likely to have been used to identify schools qualified for NHP. We compared NHP schools 
with both comparison groups and found no difference in such major school characteristics 
as baseline student performance, teacher quality, school size or location. 



HLM Models 



In this analysis, we examine the effects of the NHP on student outcomes at grade 6, 
controlling for the two student characteristics for which we have data: gender and age. The 
NHP was designed to improve literacy and numeracy, and we have therefore selected three 
outcome measures at grade 6 that are related to these objectives: student scores on the 
GSAT math test, student scores on the GSAT language test, and student scores on the 
GSAT Communications Task 1 assessment. The GSAT Mathematics and the GSAT 
Language tests are both 80-item multiple-choice tests having a maximum score of 80 
points. The GSAT Communications Task 1 is an open-ended task that is scored holistically 
on a rating scale of 1-6. The tests are not the same for 2004 and 2005, and therefore the raw 
scores should not be compared over time. In particular, the 2005 tests were based on the 
Revised Primary Curriculum, whereas those in previous years were not. We therefore refer 
to these tests as GSAT2004 and GSAT2005, to mark the difference. 

The results are organized as follows. For each of the outcome measures, we present (a) the 
null or empty model for both comparison groups, and (b) a model that tests for the effect of 
NHP on the outcome, controlling for student age and gender and for school size, location 
and the baseline 1999 school mean score on the outcome measure, also for both 
comparison groups. 

Our Level 1, or within-school, model nested students within schools and is written as: 

• + Ai -Male ■■ + ■ {Age . . - Age .) + r.. 

ij iJ iJ J iJ 

where we assume r.j ~ N{0,a^) for i = students in school j. 
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which represents, Fy, grade 6 achievement for student i in school j regressed on gender and 
age, plus the Level- 1 residual variance, , remains unexplained after student gender and 
age has been taken into account. In this model, each student’ s age is centered around its 
school mean, thus controlling for school-to-school differences in age of enrollment and 
possible repetition. Boys are coded as one and the girls as zero in the gender variable. 
Hence, the intercept, Poj, is the school-mean grade 6 achievement for girls. We treat the 
within-school gender and age gaps - the difference between GSAT scores for boys and 
girls, and for older versus younger students - as random at Level 2 to understand the 
impact of NHP on these gaps. 

At Level 2 of the model, we estimate the impact of NHP treatment on the mean grade 6 
achievement outcome in school j. We included school-level covariates to account for key 
differences among schools: school size in 1999, geographical location, and school average 
1999 GSAT score. The Level 2 model is written as: 



A. = /qq -f Yq^NHP. -f- /^^{SchoolSize J - SchoolSize) + /03 Remote j 

+ Yo 4 (GSAT99j - GSAT99) + u^j 

Aj = Fio + ruNHPj + u^j 
Pij ~ y 20 y 21^^^ j ^ 2 j 

where the mean GSAT intercept for school j is regressed on the school-level covariates, the 
NHP treatment indicator, and a residual. The within-school GSAT differences by gender 
and age given by the parameters, Pij and ^ 2 ], are specified as random, predicted by the NHP 
treatment. 

NHP Effects on GSAT Math Achievement 



The first result (table 5.2) shows that there is very little variation among schools to be 
explained; less than 10 percent of the variance in mathematics achievement for either year 
or either comparison group. There was, however, a statistically significant amount of Level 
2 variability that was potentially explainable by NHP program participation and other 
school-level characteristics. 



Table 5.2: Multilevel model partitioning variance in Mathematics, by comparison group and year 





PSM w/o replacement 


PSM with substitution 




GSAT2004 


GSAT2005 


GSAT2004 


GSAT2005 


Within-school 


211.95 


350.32 


205.71 


345.33 


Between Schools 


16.08 


28.00 


20.64 


29.00 




(7 %) 


(7.4 %) 


(9.1%) 


(7.7%) 


Chi-square 


737.86 


804.38 


897.29 


875.23 




(p<.001) 


(p<.001) 


(p<000) 


(p<.001) 
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The second result (table 5.3) shows a significant gender gap in math achievement, with 
girls outperforming boys by about 4 points in 2004 and by about 9 points in 2005. Older 
students perform less well than younger ones, a difference that is statistically significant for 
both analytic groups in 2005. The gender gap in favor of girls in Jamaica is not surprising, 
as it has been the subject of a number of studies (see, for example, Bailey 2000). It is also 
not surprising that older students perform less well than younger ones, as it is probably the 
older grade 6 students whose past performance has contributed to the grade repetition that 
is most likely responsible for being older at this level. 



Table 5.3; Multi-level model predicting Mathematics outcomes, by year and comparison group 





PSM w/o replacement 


PSM with substitution 




GSAT2004 


GSAT2005 


GSAT2004 


GSAT2005 


Fixed effect 










School mean achievement 










Intercept 


30.99**** 


43.48**** 


30.07**** 


41 .72**** 


NHP 


-0.49 


-1.76 


0.03 


-0.47 


Enrollment Gr. 1-6 


0.003** 


0.004*** 


0.004*** 


0.005*** 


Remote school 


0.21 


1.75 


0.89 


2.53** 


School GSAT1 999 


0.36**** 


0.45*** 


0.27*** 


0.43*** 


Male intercept 


-4.38**** 


-9.06**** 


-3.90**** 


-8.90**** 


Age intercept 


-0.11 


-0.35*** 


-0.19* 


-0.33*** 


NHP on male slope 


-0.08 


0.79 


-0.59 


0.70 


NHP on age slope 


-0.02 


0.24* 


.005 


0.21 



* p<.10, ** p< .05, ***P<.01, ****p<.001 



The NHP has no statistically significant impact on boosting GSAT math achievement or 
reducing the gender gap. For the GSAT2005, however, the NHP mitigates the effects of 
age on achievement. That is, even though older students in all schools perform less well 
than younger students on the GSAT2005, the difference between older and younger 
students in NHP schools is less than the difference between older and younger students in 
the matched non-NHP schools. This effect is statistically significant for the PSM without 
replacement comparison. 

Other level-2 control variables are correlated with GSAT math achievement. First, school 
level performance on the GSAT math in 1999 is positively related to student performance 
on both the GSAT2004 and the GSAT2005. Second, school size is positively related to 
math achievement, although the size of the effect is small. For each standard deviation in 
enrollment in grades 1-6, the average GSAT math score increases by 0.75- 1.50 points, 
depending on the year and comparison group. This could be due to better teacher 
qualifications in larger schools, better access to instructional material or more opportunity 
for site based training of teachers in mathematics. Remoteness is positively related to math 
achievement, but this association is statistically significant only for the GSAT2005, for the 
constructed control analytic sample, and amounts to about 13 percent of a standard 
deviation. 
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NHP Effects on GSAT Language Achievement 



The amount of variance to be explained between schools is also very small for GSAT2004 
and GSAT2005 Language scores, although for this outcome, too, there is variability to be 
explained (table 5.4). 



Table 5.4; Multilevel model partitioning variance in Language, comparison group and year 





PSM w/o replacement 


PSM with substitution 




GSAT2004 


GSAT2005 


GSAT2004 


GSAT2005 


Within-school 


225.98 


293.70 


220.96 


289.50 


Between Schools 


15.96 


23.81 


18.28 


25.03 




(6.6 %) 


(7.5 %) 


(7.6%) 


(8.0%) 


Chi-square 


661.17 


791.56 


727.76 


860.86 




(p<.001) 


(p<.001) 


(p<.001) 


(p<.001) 



Both GSAT2004 and GSAT2005 Language scores show a statistically significant and large 
gender gap favoring girls for both analytic groups. The age effect on language outcomes is 
statistically significant for both analytic groups and both years, with older students scoring 
lower than younger students. The effects are modest, however, with scores only about 15 
percent of a standard deviation lower for students who are one year (12 months) older than 
their peers. 

The effect of the NHP is counter-intuitive, as scores in NHP schools are lower than scores 
in control schools in three of the four analyses, reaching statistical significance for the PSM 
control analytic sample for GSAT2005 (table 5.5). The size of the effect is modest, about 
13 percent of a standard deviation, but the direction of the effect is not as expected. The 
NHP has no effect on reducing the gender gap. At the same time, the NHP has a positive 
effect on reducing the age gap for three of the four comparisons, and in one case - for the 
PSM with substitution control - the effect is statistically significant. 



Table 5.5: Multi-level model predicting Language outcomes, by year and comparison group 





PSM w/o replacement 


PSM with substitution 




GSAT2004 


GSAT2005 


GSAT2004 


GSAT2005 


Fixed effect 










Schooi mean achievement 










intercept 


35.04**** 


40.41**** 


34.31**** 


38.77**** 


NHP 


-0.29 


-2.33** 


0.22 


-0.77 


Enroiiment Gr. 1-6 


0.002** 


0.004*** 


0.003*** 


0.004*** 


Remote schooi 


-0.06 


2.51** 


1.03 


2.26* 


Schooi GSAT1999 


0.26*** 


0.19** 


0.20*** 


0.17* 


Maie intercept 


-6.01**** 


-9.39**** 


-6.12**** 


-8.70**** 


Age intercept 


-0.15* 


-0.26*** 


-0.26** 


-0.28*** 


NHP on maie siope 


-0.48 


0.46 


-0.40 


-0.19 


NHP on age siope 


-0.02 


0.17 


0.10 


0.19* 



* p<.10, ** p< .05, ***P<.01, ****p<.001 



The effects of other level 2 covariates are similar to those for mathematics. Students in 
larger schools score higher, as do students in remote schools on the GSAT2005. The school 
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baseline 1999 score on language is a statistically significant predictor of language scores 
for both GSAT2004 and GSAT2005. 



NHP Effects on GSAT Communications Task 1 Achievement 



The Communications Task I is a measure of basic writing competence. It is scored 
holistically, following careful rubrics, on a range of 1-6. The lower end of the scale 
represents no writing competence and the upper end of the scale represents high writing 
competence at the basic level. Again, there is relatively little between-school variance to 
explain, although there is about three times as much for GSAT2004 as for GSAT2005 
(table 5.6). 



Table 5.6: Multilevel model partitioning variance in GSAT Communications I Task, by year and 
comparison group 





PSM w/o replacement 


PSM with substitution 




GSAT2004 


GSAT2005 


GSAT2004 


GSAT2005 


Within-school 


2.77 


2.42 


2.76 


2.41 


Between Schools 


0.66 


0.14 


0.64 


0.15 




(19.2%) 


(5.5 %) 


(18.8%) 


(5.9%) 


Chi-square 


1299.33 


535.06 


1246.99 


578.23 




(p<.001) 


(p<.001) 


(P<.001) 


(p<.001) 



Again, the level 1 variables gender and age are predictors of GSAT scores, with substantial 
gender gaps for both years and analytic samples that are statistically significant for three of 
the four comparisons (table 5.7). In addition, a statistically significant age gap is observed 
for the PSM with substitution Control. The NHP has a positive effect on reducing the age 
gap in this latter group for GSAT2005. 



The effects of NHP on GSAT Communications Task I are not consistent for GSAT2004 
and GSAT2005. For both analytic samples, NHP is associated with statistically 
significantly higher GSAT2004 scores and statistically significantly lower GSAT2005 
scores. On the GSAT2004, the difference amounts to 18 percent of a standard deviation 
higher than the PSM Control and 20percent of a standard deviation higher than the PSM 
with substitution Control. On the GSAT2005, the differences are 26 percent and 13 percent 
of a standard deviation lower, respectively, than the two comparison groups. We are unable 
to explain this sign switching. 



Neither school size nor remoteness were consistently associated with GSAT 
Communications Task I scores. For GSAT2004, remote schools did have higher scores. 
Since this test is scored holistically, it is possible that standards were higher in urban and 
peri-urban schools that year, resulting in lower scores being awarded to students in these 
schools, and that in subsequent years corrections were made. 
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Table 5.7: Multi-level model predicting Communications Task I outcomes, by comparison group and 
year 



PSM w/o replacement PSM with substitution 





GSAT2004 


GSAT2005 


GSAT2004 


GSAT2005 


Fixed effect 










School mean achievement 










Intercept 


3.15**** 


1 .64**** 


3.11**** 


1 .99**** 


NHP 


0.28* 


-0.16* 


0.33** 


-0.24** 


Enrollment Gr. 1-6 


-.000 


0.00 


-0.000 


0.000* 


Remote school 


0.40** 


0.13 


0.27 


0.12 


School GSAT1999 


0.23 


0.22** 


-0.15 


-0.19* 


Male intercept 


-0.82*** 


-0.06 


-0.84**** 


-0.76**** 


Age intercept 


-0.01 


-0.01 


-0.03** 


-0.03**** 


NHP on male slope 


-0.06 


-0.05 


-0.03 


0.12 


NHP on age slope 


0.001 


0.01 


0.01 


0.01* 


* p<.10, ** p< .05, ***P<.01, 


001 









Discussion 

The remarkable negative impact of the NHP on achievement on the GSAT2005 merits 
discussion. We observe that this negative impact was not statistically significant for all 
content areas, but we do observe sign switching rather consistently between GSAT2004 
and GSAT2005. We identified three main explanations for these differences: (a) the 
scoring of the tests could have changed, (b) the population of those taking the GSAT could 
have changed, and (c) the population of those taking the GSAT in NHP schools could have 
included students who did not benefit from the program for all of primary school. We will 
consider each of these explanations. 

First, the scoring rubrics for the tests could have changed between GSAT2004 and 
GSAT2005. There is little evidence that this is the case, however. From all reports, 
considerable attention is paid to scoring the Math and Language tests objectively and using 
consistent guidelines for the holistic scoring of the Communications test. Although our 
analyses show that the GSAT2004 tests were different from the GSAT 2005, there is no 
reason to suspect this change would have a differential impact in NHP versus matched non- 
NHP schools, and indeed we found no evidence for this, as we discuss in Chapter 7. 

Second, the population of test takers could have changed significantly between 2004 and 
2005. The total number of GSAT test takers in 2005 (49196 for mathematics and 49157 for 
language) exceeded the total for 2004 (47306 for mathematics and 47306 for language) by 
approximately 1800 students, or approximately 2 students per school on average. However, 
the number of GSAT test takers in NHP schools increased relatively more, by about 3.5 per 
school, from a total of 4028 for mathematics and 4034 in language in 2004 to 4265 in 
mathematics and 4271 in language in 2005. We hypothesize that the changing composition 
of the test-taking population of students accounts for some of the differential NHP effects, 
as follows. In 2004, many principals held back some students from taking the GSAT, both 



34 




Lockheed, Harris, Gammill, Barrow and Jayasundera 



NHP Inputs, Outcomes and Impact 



to provide them another opportunity to take the test and to improve the average scores of 
the school. This practice, however, meant that the principal was obliged to take on the 
responsibility of placing the students who did not take the GSAT2004 into a suitable post- 
primary educational environment. In 2005, to avoid the additional burden of placing 
students, all or nearly all Grade 6 students were obliged to take the GSAT2005. This meant 
that some lower performing students, similar to those who had been excluded from the test 
the previous year, took the GSAT in 2005. But why would this effect be more pronounced 
in NHP schools, boosting their average GSAT2004 scores, but lowering their GSAT2005 
scores? We hypothesized that the additional attention given to NHP schools created an 
incentive for the principal to optimize the performance of students in the school in 2004, 
within the range permissible. Therefore, in NHP schools more students were excluded from 
the GSAT2004 than were excluded from the matched non-NHP schools, whose identities 
were not similarly publicly known. These excluded students took the GSAT2005, with a 
consequent depressing effect on scores in NHP schools. The number of test- takers in NHP 
schools is approximately 30 percent greater than the number of test takers in the matched 
non-NHP schools. However, the number of students in NHP schools in 2004 is nearly 40 
percent greater than the number of students in the matched non-NHP schools, which means 
that this cannot be the explanation (table 5.8). 



Table 5.8. Average number of GSAT2005 test takers per school, NHP and Matched non-NHP schools 



School type 


Test takers 
per school 2005 


NHP as % of 
comparison 


Students per 
school 2004 


NHP as % of 
comparison 


NHP 


60 


- 


410 




PSM without replacement 


46 


130 


298 


138 


PSM with replacement 


47 


128 


293 


140 


PSM with substitution 


46 


130 


300 


137 



Finally, we hypothesized that the population of the NHP schools, in particular, could have 
changed over time. We explored three ways in which this might have occurred: an overall 
increase in the number of students enrolled in NHP schools, an increase in the share of 
(lower performing) boys in NHP schools, and an increase in the average age of students in 
NHP schools. An increase in the number of students enrolled in NHP schools would 
suggest that parental choice was playing a role in determining which children attended 
these schools. In Jamaica, where multiple schools are available, parents are allowed to 
place their children in the school they perceive as more desirable. If parents were moving 
students into NHP schools throughout the primary cycle (instead of only at Grade 1), then 
many students sitting the GSAT could have experienced the NHP only partially. We do not 
have data that track individual children, but enrollment data from the School Census 
suggests that parents are selectively moving their children into NHP schools, as the average 
enrollment growth in these schools was 5 percent from 1999 to 2003, compared with no or 
even a negative increase in the matched non-NHP schools (table 5.9). We also note 
through the JASS survey that principals of NHP schools identified significantly more ways 
the public could have heard about their schools, as compared with non-NHP schools (see 
Chapter 3 and Lockheed and Jayasundera 2005). 
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Table 5.9. Average enrollment in Grades 1-6, in NHP and Matched non-NHP schools, 1999 and 2003 



School type 


Observations 


1999 


2003 


% increase (decrease) 


NHP 


72 


382 


401 


5 ** 


PSM without replacement 


70 


299 


298 


0 


PSM with replacement 


58 


298 


293 


(2) 


PSM with substitution 


66 


297 


300 


1 



t= 1.75, **p<.05 



We speculated that the increase in enrollments in NHP schools could be due to their greater 
effectiveness in retaining boys or older students, but available data do not support this 
hypothesis. Neither the share of boys taking the GSAT nor the average age of GSAT test- 
takers differed between 2004 and 2005 (table 5.10). There is one exception, and that is the 
share of boys in the PSM without replacement control schools was lower in 2004 and 
slightly higher in 2005 than it was in the other schools. 



Table 5.10. Percent male and average age in months of GSAT candidates, in NHP and Matched non- 
NHP schools, 2004 and 2005 





Percent male test-takers 


Average age of test-takers (months) 


School type 


GSAT2004 


GSAT2005 


GSAT2004 


GSAT2005 


NHP 


51.4 


52.0 


144.8 


144.6 


PSM without replacement 


48.5 


53.8 


144.6 


144.5 


PSM with replacement 


50.0 


52.0 


144.8 


144.4 


PSM with substitution 


51.1 


51.4 


144.8 


144.6 



Conclusion 



Students in NHP schools performed no higher on the GSAT than students in matched non- 
NHP schools. This absence of NHP effect was consistent across three subjects (math, 
language and communications) and for both GSAT2004 and GSAT2005. 

The age gap - that is the poorer GSAT performance of older candidates - was mitigated by 
the NHP in several cases. That is, on some GSAT2004 and GSAT2005 tests, the 
performance of older students was enhanced by being in an NHP school. 

Enrollments increased significantly in NHP schools, while falling or remaining the same in 
matched non-NHP schools. We speculate that increasing enrollments in NHP schools, 
caused by parental choice, may have resulted in some students having been exposed to a 
shorter version of the program than intended, with a consequent reduction in the possibility 
for impact. 
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CHAPTER 6. WHAT INTERVENTIONS WERE MORE 
EFFECTIVE IN BOOSTING LEARNING? 



Although the NHP as a whole did not appear to boost literacy and numeracy as measured at 
the school level for Grades 3 and 4 or at the individual level as measured by the GSAT, 
some of the interventions provided by the NHP may have been effective in improving 
student performance in weaker schools, such as those participating in the NHP. This 
chapter examines the effects of NHP-type interventions on improved learning across both 
NHP and matched non-NHP schools. The results are not as expected, and suggest 
considerable endogeneity with respect to the inputs and outcomes. 



Analytic Samples and Method 

The analysis in this chapter is based on the school average performance on the Grade 4 
Literacy test and the GSAT2005, matched with school-level responses on the JASS, for the 
71 NHP and 67 statistically matched non-NHP schools for which JASS data were 
available. We developed one or more indicators representing each of the NHP 
interventions, based on responses to the JASS. A list of interventions and the indicator 
intended to capture its degree of implementation, in both NHP and non-NHP schools, is 
provided in Table 6.1. 



Table 6.1 Intervention variables, definitions and measures 




NHP Intervention 


Definition of variable 


Measure 


1. Innovative math & literacy 


Q12 school has SIP 


Did this school develop a School Improvement Plan or School 
Development Plan for the purpose of improving the reading and 
math skills of students at the school? 


Yes = 1 


2. In-service teacher training 


Q22 Gr. 1 -3 teacher training 


About what percent of the current teachers in Grades 1-3 have 
received more than 10 days of training related to the revised 
curriculum since it was introduced in this school? 


Percent 


Q23 Gr. 4-6 teacher training 


About what percent of the current teachers in Grades 4-6 have 
received more than 10 days of training related to the revised 
curriculum since it was introduced in this school? 


Percent 


Q43 clinical supervision - literacy 


What types of school-based inservice training for literacy in 
Grades 1-6 have outside specialists provided to regular teachers 
in this school? 

Clinical Supervisory Practice 


Yes = 1 


Q48 training on reading diagnostics 


With which specific topics in literacy has the resource teacher 
helped other teachers in this school? 

Use of Diagnostic Teaching of Reading 


Yes = 1 


Q41 teacher training by outsiders 


In the past 5 years, have regular teachers of Grades 1 -6 in this 
school participated in any school-based inservice teacher training 
provided by outside specialists? 


Yes = 1 


Q42 clinical supervision - math 


What types of school-based inservice training for mathematics 
in Grades 1-6 have outside specialists provided to regular 
teachers in this school? 


Yes = 1 
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NHP Intervention 


Definition of variable 


Measure 




Clinical Supervisory Practice 




Q45 training on making math fun 


With which specific topics in mathematics has the resource 
teacher helped other teachers in this school? 

Making Mathematics Fun 


Yes = 1 


3. Governance, leadership 


Q37 SIP training 


What type of training did the principal who served the school 
during the school year 2003-2004 participate in? 

Training in School Improvement Planning 


Yes = 1 


Q37 Community mobilization wkshp. 


Workshop on community mobilization 


Yes = 1 


Q37 Principal peer learning 


Peer learning with other principals 


Yes = 1 


Q15 Local sponsor 


In the past 5 years, has any local sponsor (such as Kiwanis, 
Rotary, Lion’s Club, a local church, Adopt-a-School) helped with 
resources (cash or in-kind), either regularly or through a special 
project, for any purpose? 


Yes = 1 


4. Parent training 


Q53 governance 


What types of training have been offered to parents and other 
community members associated with this school, over the past 5 
years? 

Training in school governance and leadership 


Yes = 1 


Q53 parenting 


Training in parenting 


Yes = 1 


5. Nutrition & health 


Q62 free meal for any students 


Does this school provide a free meal (either breakfast or lunch) 
for any students? 


Yes = 1 


Q63 percent students with free 


Percent of students qualified to receive a free mean 


Percent 


meals 

6. Reading & math materials 


Q30 reading materials 


During the past 5 years, has this school received any 
supplementary reading materials for literacy? 


Yes = 1 


Q31 number of materials 


What sorts of supplementary reading materials did this school 
receive? 


Number of 
materials 


Q34 math materials 


During the past 5 years, has this school received any 

supplementary mathematics materials? 


Yes = 1 


Q35 number of math materials 


What sorts of supplementary math materials did this school 
receive? 


Number of 
materials 


7. Computers 


Q65 computers for student use 


Does this school have computers for student use? 


Yes = 1 


Q66 internet access 


Does this school have internet access? 


Yes = 1 


Q69 computers for administration 


Is there a functioning computer available for school records and 
other administrative purposes? 


Yes = 1 


8. Resource teachers 


Q40 training for math 


During the last school year, 2004-2005 including the summer, 
about how many days of training did the math Support Teacher 
receive? 


Number of 
days 


Q40 training for reading 


During the last school year, 2004-2005 including the summer, 
about how many days of training did the reading Support Teacher 
receive? 


Number of 
days 


9. & 10 EMIS & Data Bases 


Q70 JSAS installed 


Q70. Has the Jamaica School Administrative System (JSAS) 
software been installed in a computer at this school? 


Yes = 1 



We used ordinary least squares (OLS) multivariate regressions and estimated the effect of 
the interventions on five outcome variables: Grade 4 Literacy non-mastery, Grade 4 
Literacy less-than-mastery (that is, non-mastery plus partial mastery), GSAT2005 
mathematics, GSAT2005 language and GSAT2005 communications I scores. The two 
Grade 4 measures are the percent of students in the school attaining non-mastery and the 
percent of students in the school attaining less-than-mastery on the 2004 test 
administration. The three GSAT2005 measures are school average scores on each test. 

The models vary with respect to the outcome measure, and include as predictors only those 
inputs directly related to the outcome. Thus, inputs related to literacy are included in the 
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literacy and language models and inputs related to math are included in the math model. 
Specifically, we include “supplementary reading materials” and teacher training for literacy 
and reading in the literacy, language and communications regressions, but not in the math 
regression. We include “supplementary math materials” and teacher training for math in the 
math regression but not in the literacy, language or communications regression. In all 
regressions, we control for the school’s geographical location (rural or urban) and for the 
GSAT2005 regressions we control for the school average score on the GSAT1999, for each 
test separately. The results are presented in columns 2-6 of table 6.2. 



Table 6.2 Factors affecting school average literacy and numeracy scores, in NHP and matched non- 

NHP schools, controlling for urban rural and 1999 test scores (GSAT models only) 

Grade 4 Literacy GSAT 2005 



NHP Intervention 


Non- 

mastery 


Less than 
Mastery 


Math 


Language 


Commi 


1. Innovative math & literacy 


Q12 School has SIP 


-0.083 


-0.167 


0.875 


1.886 


-0.015 


2. In-service teacher training 


Q22 Gr. 1 -3 teacher training 


-0.001 


0.00 


- 


- 


- 


Q23 Gr. 4-6 teacher training 


- 


- 


-0.029* 


-0.044*** 


-0.003 


Q43 clinical supervision -literacy 


0.035 


0.008 


- 


-0.157 


-0.177* 


Q48 training on reading diagnostics 


-0.035 


-0.036 


- 


-0.165 


0.103 


Q41 teacher training by outsiders 


0.021 


0.029 


1.487 


-0.256 


0.021 


Q42 clinical supervision - math 


- 


- 


-0.311 


- 


- 


Q45 training on making math fun 


- 


- 


-1 .584 


- 


- 


3. Governance, Leadership 

Q37 SIP training 


-0.012 


-0.023 


1.3 


0.72 


0.097 


Q37 Community mobilization wkshp. 


0.025 


0.07 


-1.275 


-0.301 


-0.082 


Q37 Principal peer learning 


0.024 


0.011 


-2.67* 


-2.635** 


-0.048 


Q1 5 Local sponsor 


0.025 


-0.002 


-2.319 


-2.626* 


-0.1 


4. Parent training 


Q53 governance 


-0.008 


-0.019 


2.804 


2.448 


0.053 


Q53 parenting 


-0.011 


-0.047 


-0.079 


0.026 


-0.085 


5. Nutrition & Health 


Q62 free meal for any students 


0.031 


-0.296 


6.761 


7.303 


0.58 


Q63 percent students with free meals 


-0.002 


-0.003 


-0.293 


-0.131 


-0.011 


6. Reading & math material 


Q30 reading materials 


-0.035 


-0.006 


- 


1.87 


0.348 


Q31 number of materials 


-0.024** 


-0.03* 




0.528 


0.024 


Q34 math materials 


- 


- 


1.767 


- 


- 


Q35 number of math materials 


- 


- 


0.35 


- 


- 


7. Computers 


Q65 computers for student use 


0.018 


0.007 


2.447* 


1.312 


0.06 


Q66 internet access 


-0.019 


-0.042 


3.641** 


4.325*** 


0.22 


Q69 computers for administration 


-0.022 


-0.116* 


5.086** 


4.578** 


0.411* 


8. Resource teachers 


Q40 training for math 


- 


... 


0.202 


- 


- 


Q40 training for reading 


-0.005* 


-0.004 


- 


0.322*** 


0.02 


9. & 10 EMIS & Data Bases 

Q70 JSAS installed 


0.047 


0.078 


-2.595 


-1.943 


-0.259 



Control variables 

GSAT 1 999 Score 
Urban 


0.036 


0.083 


0.367*** 

-1.119 


0.143 

-0.585 


0.145 

-0.146 


Constant 


0.305 


1.152*** 


17.599** 


19.064*** 


0.32 


Observations 


118 


118 


116 


116 


116 


Adj. R-squared 


0.119 


0.01 


0.195 


0.219 


0.028 


Legend: * p<.10; ** p<.05; *** p<.01 

Note: (-) indicate variables not included in model. 
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Curious Results on the Eeeects oe the Interventions 

Each regression model includes at least 20 independent variables; with a 5 percent level of 
statistical significance, we would expect to find by chance at least one “statistically 
significant” variable in each regression, and with a 10 percent level of statistical 
significance we would expect to find at least two “significant” variables. To avoid dwelling 
on spurious findings, therefore, the discussion of results will focus on those variables that 
show a consistent pattern of statistical significance across the outcome measures. All 
regression results include some variables with coefficients whose signs are in the opposite 
direction from that expected, which requires further exploration. 

Grade 4 Literacy 

The Grade 4 non-mastery and less-than-mastery regression models explain very little 
variance in these measures, with adjusted R-squares of .12 and .01 respectively (Table 6.2 
columns 1 and 2). Only two variables were statistically significant in the model predicting 
the percent of students in the school scoring non-mastery; these were the number of types 
of supplementary reading materials received by the school and resource teacher 
training for reading. The simple correlations between these measures and Grade 4 non- 
mastery was -.28 and -.12, showing that schools with more types of supplemental reading 
materials and more resource teacher training also had smaller shares of students scoring at 
the non-mastery level on the Grade 4 Literacy test. At the margin, for each additional type 
of reading material received by the school, the percent of students in the school scoring at 
the non-mastery level dropped by nearly 2 percentage points; each additional day of 
resource teacher training lowered the share even further. 

Two variables were statistically significant (at the 10 percent level of significance) in the 
model predicting the percent of students in the school scoring less-than-mastery; these were 
supplementary reading materials and the availability of a functioning computer for 
school records. The simple correlations between these measures and less-than-mastery 
were -.17 and -.07, respectively. At the margin, each additional type of reading material 
was associated with a 3 percent decline in the number of students performing below 
mastery level, while schools with a functioning computer for school records had nearly 12 
percent fewer students scoring at the less-than-mastery level. Given that these variables 
could have emerged as statistically significant by chance, these results should not be over- 
interpreted. 

GSAT2005 

The GSAT mathematics and language regression models are more robust, with adjusted R- 
squares of .19 and .22 respectively (Table 6.2 columns 3 and 4). For GSAT2005 
mathematics, controlling for GSAT1999 math, five variables are statistically significant in 
the model predicting the school average scores, two of them, however, have effects in the 
direction opposite to that expected. These are teacher training and principal peer learning. 
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with simple correlations with GSAT2005 math of -.06 and -.08, respectively. At the 
margin, for each additional percent of teachers in Grades 4-6 who received more than 10 
days of training in the revised curriculum, GSAT2005 school average math scores declined 
by .03 points, which is not meaningful. Also at the margin, schools whose principals 
participated in peer learning scored 2.8 points lower on the GSAT2005 math test, or 
approximately 15 percent of the GSAT2005 standard deviation, which is meaningful. 

Computers and internet access appear to boost math achievement, with simple correlations 
with GSAT2005 math of .17 and .29, respectively. At the margin, schools with 
functioning computers for student use achieve GSAT2005 math scores 2.4 points higher 
than those without computers. In addition schools with internet access achieve GSAT2005 
math scores 3.6 points higher than those without internet access. Finally schools with a 
functioning computer for administrative purposes also score higher - by over 5 points. 
Thus, we estimate that schools with all three computer inputs would score over 1 1 points 
higher on the GSAT2005 math test, for an effect size of more than 60 percent of the 
standard deviation. This is both substantial and meaningful. 

Six variables are statistically significant for boosting GSAT2005 language scores, with 
three having effects in the opposite direction to the expected. These are teacher training, 
principal peer learning, and local sponsorship, with simple negative correlations of -.12, - 
.12 and -.04, respectively. At the margin, for each additional percent of teachers in Grades 
4-6 who received more than 10 days of training in the revised curriculum, GSAT2005 
school language scores declined by .04 points, which is not meaningful. Also at the margin, 
schools whose principals participated in peer learning scored 2.8 points lower on the 
GSAT2005 language test, also approximately 15 percent of the GSAT2005 standard 
deviation, which is meaningful. Finally, also at the margin, schools having local sponsors 
that provided resources scored 2.6 points lower on the GSAT2005 language test, also a 
meaningful difference. 

Three other variables, however, are associated with higher scores. These are resource 
teacher training for reading and two of the computer variables discussed above. At the 
margin, each additional day training received by reading resource teachers boosted 
GSAT2005 language scores about one-third of a point. And the effects for having 
computers and internet access are similar to those for math, albeit slightly lower in their 
combined effect. 

For the GSAT2005 Communications I test, two variables were statistically significant (at 
the .10 level), with one in the opposite direction from that expected. Schools reporting 
more teacher training had lower average scores, although the difference is not meaningful. 
And schools with computers for administrative purposes scored higher, by about .41 
points. 

Table 6.3 presents the regression coefficients for the six variables that are statistically 
significant in more than one of the five regression models. Supplementary reading 
materials, training of reading resource teachers, computers for administrative purposes and 
internet access are associated with higher scores at the margin, while teacher training and 
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principal peer learning are associated with lower GSAT2005 scores at the margin. It 
appears that certain types of training and provision of resources are helpful, while other 
types of training are not helpful. What explains these differences? 

Table 6.3 School input factors related to literacy and numeracy in two or more models 





Gr. 4 Non- 
Mastery 
(percent 
students) 


Gr. 4 Less- 
than-Mastery 
(percent 
students) 


GSAT2005 

Mathematics 

(school 

average) 


GSAT2005 

Language 

(school 

average) 


GSAT2005 
Communications 1 
(school average) 


Teachers in Grades 4-6 
who received more than 
1 0 days of training in the 
revised curriculum 






-0.029 


-0.044 


-0.003 


Principals participated in 
peer learning 






-2.670 


-2.635 




Resource teacher training 
in reading 
Number of different 
supplementary reading 
materials 


-.005 

-.024 


-.03 




.322 




Functioning computer for 
administrative purposes 




-.12 


5.086 


4.578 


0.411 


Internet access 






3.641 


3.325 





Source: Table 6.2 



What Accounts for These Results? 

The most plausible explanation for the curious results described in the previous section is 
endogeneity. That is, schools received resources, from whatever provider, in response to 
need for intervention, as indicated by lower student performance on the Grade 4 Literacy 
and GSAT tests. Teachers in poorly performing schools received comparatively more in- 
service training, possibly because they teachers in these schools were themselves less well 
trained, or because the students presented more challenges to the teachers, or because 
supervisors judged that additional training was needed for other reasons. Similarly, 
principals of schools where students’ performance was lower may have sought out peer 
learning opportunities with other principals of similar schools, in an effort to deal with 
these common challenges. We interpret this to mean that the service providers recognized 
the need for in-service training in weaker schools, and ensured that these schools received 
comparatively more of this input. 

We find a consistent positive effect for computers on achievement. While it is possible that 
computers and internet access actually helped boost student learning, it is also possible that 
computer resources were provided to better performing schools because they also had 
better leadership and initiative. Under the NHP program, for example, schools needed to 
request computers and demonstrate that they had the capacity to retain them in a secure 
environment. The schools also needed to have a regular source of electricity. Thus, better 
schools - including those with electricity and school-level initiative - were more likely to 
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receive computers. We included initial student performance as measured by the GSAT1999 
in the regression estimates to control for school quality at the outset, but these statistical 
controls are unlikely to have captured all the quality dimensions of the school. Other, 
unmeasured “school quality” factors may have affected which type of services the schools 
received. Specifically, we have no independent measure of instructional leadership at the 
school that could help explain why some schools, but not others, received computers. 

We also find a positive effect for reading resource teacher training and variety of 
supplementary reading materials. Endogeneity does not easily explain these results, unless 
obtaining these materials and services is also related to leadership at the school level. 
Resource teachers at the school level seem to help other teacher improve their skills, so that 
fewer students score at the non-mastery level on the Grade 4 Literacy test and students 
achieve higher language scores on the GSAT. Supplemental reading materials also reduce 
the share of children scoring below mastery level on the Grade 4 Literacy test, possibly 
being used by classroom teachers with support from trained reading resource teachers. 

Conclusions 



In Chapter 3 we noted that the NHP schools were generally more advantaged with respect 
to NHP type of inputs than were the comparison schools. In this chapter we find that these 
advantages did not in general translate into higher scores on tests of literacy and numeracy. 
However, inputs of teacher and principal training in the schools appear to have been 
targeted at schools in need of such interventions. With respect to computers, resource 
teacher training and supplementary reading materials, it is plausible that schools with better 
leadership were more active in securing these resources. 
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CHAPTER 7: USE OF NATIONAL TESTS TO MONITOR 

CHANGE OVER TIME 



This chapter assesses the utility of four Jamaican MOEYC tests as indicators for 
monitoring change over time, with particular reference to USAID PMP indicators. These 
are: the Grade 1 Readiness Inventory, the Grade 3 Diagnostic Test, the Grade 4 Literacy 
Test, and the Grade Six Achievement Test. We conclude that due to anticipated MOEYC 
policy changes in test administration dates and content^, it is unlikely that USAID will be 
able to rely on a year-to-year criterion-based impact analysis. We recommend that USAID 
monitor project impact by comparing the performance of project schools relative to an 
empirically derived comparison group of schools, the approach used in the Impact of New 
Horizons for Primary Schools on Literacy and Numeracy in Jamaica, 1999-2004 
(Lockheed, Harris, Gammill and Barrow 2005). 

Grade 1 Readiness Inventory 



The projected utility of the Grade 1 Readiness Inventory depends on the outcomes of 
pending MOEYC decisions regarding the timing and administration of the assessment. If 
the Inventory remains a low stakes test administered by grade 1 teachers, it could provide a 
useful baseline for assessing student entry-level skills and assuring the comparability of 
project and comparison schools. 

If the MOEYC decides to change current policy and the Inventory administration is 
conducted before school begins by preschool teachers, there will be pressure for the test to 
become high stakes; administration quality and reporting response rate may be jeopardized, 
the need for annual test development will be costly and require equating in order for results 
to be comparable from year to year; during this transition, year-to-year results will be 
compromised. 

Regardless of the MOEYC decision, one limitation of the current procedures is that only 
mastery level information is available for analysis. Actual scores on each subtest would 
provide more precise information and a more sensitive assessment of student skills. Actual 
student scores should be collected from project and comparison schools. With this 
information, the indicator would refer to average scores on each the 4 subtests and overall 
score (Visual Motor, Visual Perception, Auditory Perception, and Number/Letter 
Knowledge, Total). 



^ Refer to Technical Support Provided to MOEYC on Pending Decisions for Student Assessment in the 
October, 2005 status report for a more complete description and analysis of the pending decisions. 



44 




Lockheed, Harris, Gammill, Barrow and Jayasundera 



NHP Inputs, Outcomes and Impact 



Grade 3 Diagnostic Test 



In the absence of the proposed MOEYC changes, average subtotal scores in Language Arts, 
Mathematics and Communication on the Grade 3 test would appear to be good indicators 
of learning . The test is a low stakes test. The current reporting form includes raw subtest 
scores, there is a relatively high response rate (85%), and the test blueprint has been revised 
to reflect the RPC. It should be noted that alignment with the RPC is new and data from 
past years may not be comparable. In addition, caution should be used in interpreting the 
math subtests without first taking into account reading proficiency (See Note at end of 
chapter 1). 

If a change in the timing of the test occurs, the utility of the Grade 3 exam as a PMP 
indicator will depend upon when the change occurs and how it is implemented. Unless the 
MOEYC administers the test to two cohorts in the same calendar year, one cohort will not 
have data. In addition, the comparability of the year-to-year scores will be affected. It is 
likely that the test content/difficulty would be adjusted and, even if the test remained the 
same, it is not equivalent to compare average scores from children who took the test at the 
end of Grade 3 with scores of those who took the test at the beginning of the year. 

To support its utility as a PMP indicator and to enhance the test interpretation, post-testing 
scaling of Grade 3 test results is recommended to assure the comparability of year-to-year 
scores. To do this, the MOEYC would need to systematically collect item data from a 
sample of schools. External technical support will be needed on a short-term basis. 



Grade 4 Literacy Test 



The Grade 4 Literacy could be a useful indicator for monitoring learning of the youth 
targeted by Strategic Objective 12. It is designed to identify the lowest performing students 
in relation to stable literacy criteria. The return rate is high. However, the utility of the 
Grade 4 Literacy exam as an indicator is limited because the reporting sheet only includes 
mastery information. No subscores or total scores are provided. Students are categorized as 
mastery /non-mastery in each of the three areas and then given an overall rating of Mastery 
(mastery on all three areas). Almost Mastery (mastery on 1 or 2 areas), or Non Mastery 
(non-mastery on all 3 areas). This lack of precision in what is reported significantly limits 
the sensitivity of the Grade 4 test for evaluation and monitoring purposes. Eor example. 
Mastery in word recognition requires at least 75% correct. On the reporting form, a student 
who had none correct is indistinguishable from a student with 74% correct. In addition. 
Grade 4 repeaters are not identifiable. 

In order to use the Grade 4 Literacy test data effectively, USAID would need to support 
collection of subscale and repeater data. In the short term it is likely that this will need to be 
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done on a school-by-school basis for all project and comparison schools. In the long term, 
it is recommended that resources be made available to support the revision of the score- 
reporting sheet to include actual student scores on each of the subtests and repeater 
information. If the subscore and repeater data are available, average subscores for non- 
repeaters (and, if sufficient numbers, average subscores of repeaters) could serve as useful 
indicators. 



Grade 6 Achievement Test (GSAT) 



The Grade Six Achievement Test (GSAT) is a high-stakes, curriculum-based exam 
administered nationally during sixth grade to determine students’ secondary school 
placement. In the past, average performance on the GSAT has been used by USAID as one 
of the indicators for monitoring learning. 

In 1999, the GSAT replaced the Common Entrance Exam as the “gate-keeper” for entry to 
all secondary school schools including the elite schools; currently, the GSAT “places” 
virtually all students sitting the exam into some secondary program. The most demanding 
placement decisions, however, occur at the top of the scale, to distinguish among high 
scoring students for places into the few prestigious high schools. Consequently, the test has 
tended to focus on measuring student performance with greater precision at the top of the 
scale. In one year, additional harder items were added to the GSAT to assure accuracy of 
decisions affecting high scorers. 

Effective with the GSAT2005, the test blueprint and content are aligned with the Revised 
Primary Curriculum (RPC). Test structure and subject areas covered remain the same as 
earlier versions of the GSAT: mathematics (80 multiple choice items), language (80 
multiple choice items), science (60 multiple choice items), social studies (80 multiple 
choice items), and communication (2 writing tasks). According to test developers from the 
Student Assessment Unit, the primary differences between the content of past GSAT exams 
and GSAT2005 are in science and social studies where new topics have been added or 
rearranged. 

GSAT as a Monitoring Indicator 



Eor the GSAT to be an effective indicator for monitoring changes in learning from one year 
to the next, it must meet the following criteria: (1) the test content must be stable, (2) the 
test must be adequately precise for students of various ability levels including low 
performers, and (3) the scores must be equated or anchored to a common scale. 

Stability of test content: In recent years the MOEYC has worked to achieve alignment 
between the RPC and the rest of the educational system. The New Horizons Project 
included support for the RPC. Teacher training colleges received support to align their 
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curriculum and teacher certification process to the RPC. Similarly, the Student Assessment 
Unit has worked to revise test specifications to the RPC and to develop items that support 
the revised specifications. Since the test blueprint for the GSAT has been revised to reflect 
the RPC, it seems likely that the test content specifications will remain fairly stable for the 
near future. 

Measurement precision for students of various ability levels: Several strategies were 
used to evaluate how well the GSAT2004 and GSAT2005 tests of mathematics and 
language functioned for students of different ability levels. Since USAID interventions are 
typically targeted at raising the math and literacy achievement of lower performing 
students, USAID monitoring indicators need to be sensitive to small changes in the 
performance of low scorers as well as in changes overall. As a first step, the score 
distributions for GSAT2004 and GSAT2005 were compared separately for math and 
language tests, for all test-takers in Jamaica. Second, item analyses were conducted to 
identify those items that functioned better for low ability students. Third, these “easier” 
items were aggregated into “new” math and language subscores and reanalyzed. 

A comparison of GSAT2004 scores with GSAT 2005 scores indicates that the test changed 
dramatically between the two years. Across all test takers in Jamaica, the average score for 
math jumped over 1 1 points while the average score for language jumped 4 points. Figures 
7. 1-7.4 illustrate the score distributions for GSAT2004 and GSAT2005 for mathematics 
and language. Each bar in the figure indicates the number of students scoring within the 
score range specified on the horizontal axis. As is evident from the figures, the GSAT2004 
results were positively skewed. Low performers tended to be lumped together within a 
relatively small range of scores whereas high performers were spread out over a wider 
range of scores (i.e., there was better discrimination between high scorers). Score 
distributions for GSAT2005 are much more evenly distributed across the scale. While there 
are still large numbers of students scoring at or just above the change level (students who 
guess on all of the 80 multiple choice items should get a score of about 20 or about 14 of the 
items correct by chance), numbers of students scoring at other points on the scale are more 
evenly distributed. 

How well does the test function for low performers? Item analyses using classical test 
theory and item response theory methodologies confirmed that GSAT2005 was better able 
to discriminate amongst lower ability students than was GSAT2004. As indicated in Table 
1, fewer test-takers scored at the chance level on GSAT2005 than on GSAT2004. In 
addition, the reliability of the scales for students scoring below the median was quite low 
for GSAT2004 (coefficient alpha for Math = .58 and for Language = .47) as compared with 
GSAT2005 (coefficient alpha for Math = .81 and for Language = .68). 
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Figure 7.1: Distribution of scores 



Figure 7.2: Distribution of scores 



GSAT 2004 Math 



GSAT 2004 Language 




Figure 7.3: Distribution of scores 




Figure 7.4: Distribution of scores 



GSAT 2005 Math 




0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 

5.0 15.0 25.0 35.0 45.0 55.0 65.0 75.0 



GSAT 2005 Language 




0.0 1 0.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 

5.0 15.0 25.0 35.0 45.0 55.0 65.0 75.0 
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Although GSAT2005 functioned better for low performers than GSAT2004, there were 
still sizable proportions of the students scoring close to the chance level. We wondered if 
the GSAT2005 could be used to measure performance more precisely for those students 
scoring around the chance level, by constructing “easier” tests from the existing items, even 
though we did not have access to the actual wording of the items and did not collect 
additional data. We used a statistical method to identify items that performed better for the 
lower-performing test-takers. Identified items satisfied the following criteria: (a) adequate 
overall item indices, and (b) relatively better (>.2 item to total correlations) discrimination 
for scorers below the median. These items tended to be easier items. Using just these items, 
we generated “easier” Math and Language tests and rescored the tests, to create “easy” 
GSAT2005 math and language scores for all test takers. These new scores are based solely 
on an empirical analysis of the item data from the national administration. As evident in 
Table 7.1, the new versions of GSAT2005 had slightly higher reliability for lower 
performing students without significantly reducing the reliability for all students. 

Curiously, although the new Math test reduced the percentage of students scoring below 
the chance level, the new Language test did not. Possibly students with questionable 
literacy will not perform better on the Language test regardless of how easy it is. 



Table 7.1: Test statistics for GSAT2004 and GSAT2005 Mathematics and Language tests and an 
empirically generated easier version of the GSAT2005 tests 





GSAT2004 


GSAT2005 


GSAT2005 
Easier version* 


Math 

Mean (SD) 


34.91 (16.52) 


46.23 (18.82) 


(Based on 42 items) 
28.43 (10.65) 


Median 


30 


47 


31 


Skewness 


.742 


-.054 


-.528 


N 


47396 


49196 


49196 


Reliability (all students) 


.95 


.96 


.94 


Percentage of students scoring at or 


19.5 


9.7 


6.8 


below chance (Score<25% correct) 


Reliability for students scoring below 


.58 


.81 


.84 


the median 


Language 
Mean (SD) 


38.38 (16.38) 


42.59 (17.98) 


(Based on 35 items) 
21.67 (9.50) 


Median 


35 


41 


23 


Skewness 


.451 


.166 


-.202 


N 


47416 


49157 


49157 


Reliability (all students) 


.94 


.95 


.94 


Percentage of students scoring at or 


13.1% 


11% 


14% 


below chance (Score<25% correct) 


Reliability for students scoring below 


.47 


.68 


.73 


the median 



Scores from these “easier” versions were hypothesized to be more sensitive for identifying 
smaller differences in NHP and their comparison schools. Thus, Math and Language scores 
using just the items from the “easier” versions were calculated for all students and then 
used as dependent measures in supplemental analyses repeating the OLS analyses that had 
been conducting using the full tests. Findings replicated those reported in Chapter 4 
suggesting that even with a more sensitive achievement measure, performance in NHP and 
matched non-NHP was not significantly different. 
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Scores must be equated or anchored to a common scale: As is evident from the 
previous analyses, the GSAT changed dramatically between 2004 and 2005. This is likely 
due in part to a change in the blueprint and test specifications required to align the test to 
the RPC. However, at this time, the SAU does not equate scores from one year to the next 
and over the last several years the scores have fluctuated considerably (see Lockheed, 
Harris, Gammill and Barrow 2005 for details). Thus, an 11-point increase in average math 
scores between GSAT2004 and GSAT2005 does not mean that students performed better 
in 2005. Rather, changes in average performance reflect changes in the test and may also 
reflect changes in the student candidate pool. Although the SAU follows test development 
procedures to create parallel forms of the GSAT by using test blueprints, item 
specifications, and pretested items, they do not conduct analyses following test 
administration to equate scores for year-to-year comparisons. In past years they have 
repeated anchor items in order to facilitate equating. At this time it is unclear if these items 
would be adequate to begin the equating process for GSAT2006. However, in order for the 
GSAT to be useful eventually for year-to-year monitoring, this equating process would be 
critical. It would be especially useful to articulate a scale with meaningful score ranges and 
then to anchor the test each year against this common scale. The SAU is interested in this 
technical support but they do not currently have the skills in-house to conduct the scaling. 



Conclusion and Recommendations 

To be used to monitor change over time, all existing national tests would need to be 
horizontally equated. Test development processes are in place that would enable this to be 
done, but post- test equating is not currently practiced for any of the national tests. 

Given the uncertainty of the possible changes in content and timing of the Grade 3 
Diagnostic test, its utility for use as a monitoring indicator is questionable. 

The Grade 4 Literacy test holds promise as a valuable measure for monitoring the literacy 
skills of low performing Jamaica youth. It would be necessary to collect actual Grade 4 
Literacy test item responses and scores, not just mastery levels as is the current practice. 
The GSAT holds promise as a potential PMP indicator. The content of the test is likely to 
be stable over the next few years. The distribution of scores and the item analyses of the 
GSAT2005 tests of Math and Language suggest that the newly revised test blueprint targets 
skills across the score range and not just at the top of the range, as was the case for 
GSAT2004 and before. 

The major drawback for using the GSAT as a PMP indicator is that the test is not equated 
from year to year, and thus provides no common scale against which annual fluctuations in 
performance can be understood and adjusted. 

Thus, the GSAT can be meaningfully used to compare students and schools within a 
given year, but GSAT scores cannot be meaningfully compared from one year to the 
next. It is strongly recommended that the SAU be provided with technical support to 
facilitate and conduct the equating process. 
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Note to chapter 7 

A review of mathematics items on the 2004 Grade 3 Diagnostic Test indicated that almost 
all items required reading comprehension (see table below). This is not a critique of the 
items. Current best practice in numeracy education emphasized mathematics in the context 
of problem solving and practical applications. Jamaica began implementing this approach 
as part of its primary curriculum reform in the 1990s. It is appropriate to measure 
mathematics that is contextualized. However, students who are poor readers will be 
disadvantaged in reading comprehension should be considered in interpreting the math 
scores. 



Area 


Item 

numbers 


Number of items 


Share of test 
(percentage) 


Number of words 
per item (average) 


Number 


1-30 


30 


50 


10 


Estimation and Measurement 


31-40 


10 


17 


10 


Geometry 


41-47 


7 


12 


9 


Algebra 


48-54 


7 


12 


4 


Statistics 


55-60 


6 


10 


20 
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CHAPTER 8. CONCLUSIONS AND RECOMMENDATIONS 



The New Horizons for Primary Schools (NHP) was implemented in 72 government schools 
in Jamaica, from 1998-2005. The program provided support to schools on the basis of 
needs identified through the preparation of a School Development Plan (also called a 
School Improvement Plan). This independent evaluation report first compares the schools 
in the NHP with a statistically matched comparison group of government schools, with 
respect to (a) support received for literacy and numeracy, (b) learning outcomes as 
measured by national tests in grades 3, 4 and 6, and (c) impact of NHP on learning comes. 
Second, the report examines school-level factors related to gains in literacy and numeracy. 
Third, the report assesses the utility of the existing national tests for monitoring change 
over time. Finally, the report makes recommendations for better measuring NHP outcomes 
and for better assessing the impact of the NHP on student learning. 



Conclusions 



Support for literacy and numeracy was greater in NHP schools as compared with a 
group of statistically matched non-NHP schools. In October and November, 2005, 
researchers visited 71 extant NHP and 67 statistically matched comparisons schools and 
carried out group interviews with the principal and experienced teachers. The survey asked 
about innovative mathematics and literacy programs at the school, in-service teacher 
training, governance and leadership training, parent education and training, supplementary 
reading and math materials, computer use in schools and training teachers about computers, 
training resource teachers, nutrition and health programs, integrating statistical data bases, 
and linking project schools with national EMIS. In all areas, other than parent education 
and health and nutrition, NHP schools were more advantaged than matched non-NHP 
schools. 

Learning outcomes in NHP schools and matched non-NHP schools fluctuated over 
time along with national fluctuations in test results and were not consistently higher 
in NHP schools as compared to matched non-NHP schools. National school-level 
performance data on the Grade 3 Diagnostic test and the Grade 4 Literacy test and 
individual student performance data on the Grade 6 Achievement test (GSAT) were 
obtained and analyzed. No differences were observed in seven of the ten tests analyzed. 

The performance of students in NHP schools was higher than the performance of students 
in the statistically matched non-NHP schools on two tests: 

• 2004 Grade 3 Communications Task I 
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• 2004 Grade 6 Communications Task I 

The performance of students in NHP schools was lower than the performance of students in 
the statistically matched non-NHP schools on one test: 

• 2005 Grade 6 Communications Task I 

The performance of students in NHP schools was no different from the performance of 
students in the statistically matched non-NHP schools on the following seven tests: 

• 2004 Grade 3 Math 

• 2004 Grade 3 Language 

• 2004 Grade 4 Literacy 

• 2004 Grade 6 Math 

• 2004 Grade 6 Language 

• 2005 Grade 6 Math 

• 2005 Grade 6 Language 

It appears that certain types of training and provision of resources are helpful, while 
other types of training are provided in response to school need. At the margin, 
supplementary reading materials, training of reading resource teachers, computers for 
administrative purposes and internet access are associated with higher scores, while teacher 
training and principal peer learning are associated with lower GSAT2005 scores. 

The existing national tests have drawbacks that limit their utility as USAID PMP 
indicators. Each of the national tests was analyzed to assess their utility as monitoring 
indicators. Drawbacks differ from test to test, and the principal ones are: 

• Possible changes in the timing and content of the Grade 3 Diagnostic test 

• Lack of precision in reporting sub-scores and total scores from the reporting sheet 
of the Grade 4 Literacy test, which reports only non-mastery, near-mastery or 
mastery levels of performance 

• Lack of uniformity in forwarding to the MOEYC the Grade 3 and Grade 4 tests 
results; using these tests for monitoring would require improved data collection, 
which is ongoing. 

• Absence of centrally-available student-level test score data for the Grade 3 and 
Grade 4 assessments. In addition, student-level demographic data, including age 
and gender, and student- level school history data, such as the student’s attendance, 
past performance on tests, or classroom assignments, would improve precision of 
monitoring and evaluation analyses. 

• Eack of annual, post-test horizontal equating of Grade 3 and Grade 4 tests, through 
empirical IRT methods, to place test results on a common scale to ensure 
comparability of measures over time. 
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• Asa placement examination, the GSAT assessment is not designed to measure 
student performance at the lower end of the scale, although the GSAT2005 has 
greater discrimination at this level. Since NHP schools were selected because the 
average performance of their students is at the low end of the scale, this is 
problematic. 

• The GSAT tests are also not horizontally equated (so there is considerable 
fluctuation from year to year). As a result, trend data are not meaningful and year 
to year comparisons are inappropriate. 

• Waiting five or six years to measure improvement using the GSAT is too long. 
Using the Grade 3 Diagnostic Test and the Grade 4 Literacy Assessment would 
provide better interim measures. 



Recommendations 



Monitoring and evaluation could be strengthened, to better measure the impact of the 
NHP and provide guidance for system improvement. For better monitoring of changes 
over time, the AED Team recommends that the USAID Mission work with the MOEYC 
and with external technical assistance to ensure that: 

• The GSAT has fidelity at the low end of the scale and is horizontally equated. 

• The GSAT, Grade 3 Diagnostic and Grade 4 Eiteracy tests are horizontally equated. 

• Actual Grade 4 Eiteracy test item responses and scores are reported centrally, not 
just school averages of mastery levels 

Eor better measuring the impact of the NHP, we also recommend that the USAID Mission 
work with the MOEYC and with external technical assistance to ensure that: 

• Prior to implementing the new NHP, a prospective comparison group is created 
through propensity score matching (PSM) of all NHP schools with a sample of non- 
NHP schools, using the most recent data from the Annual School Census and 2005 
national test results to create the propensity scores. The identification of control 
schools at the outset is essential, given that year-to-year comparisons are not 
meaningful with the existing, un-equated tests. 

• The ISAS software is available, functional and staff trained in the use of this 
software. The software would serve as a school based tool and would also serve the 
needs of USAID and the MOEYC. This software would need to be installed in all 
NHP and matched comparison schools, and computers provided as needed to these 
schools (to replace obsolete computers or to provide computers for administration). 
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Data from the school based JSAS would need to be collected on a regular basis 
from all NHP and matched comparison schools. 

• Data from all NHP and all statistically matched comparison schools are regularly 
collected. These data would cover the implementation of specific aspects of the 
NHP or provision of similar inputs to matched comparison schools. 

• Independent evaluators carry out the program evaluation, including (a) extracting 
routine administrative data, (b) collecting any additional data as necessary, and (c) 
using statistical methods to assess the impact of the NHP program as a whole and 
evaluate the contribution of various interventions to improvements in learning 
outcomes. 
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ANNEX A. PROPENSITY SCORE MATCHING 



In the earlier report (Lockheed et al 2005), propensity score matching (PSM) was carried 

-j 

out using a data set in which four errors of coding were subsequently identified. 

Corrections were made in the identification of four schools with respect to their status as 
NHP schools, in the matching of School Census data with GSAT data at the school level 
for one school and in estimating a 1999 GSAT score for one school, based on a regression 
where GSAT1999 = f (GSAT2000). 

One consequence of these corrections was that the PSM needed to be re-estimated for the 
72 NHP schools. We did this, using nearest-neighbor matching with calipers of .1, .05 and 
.01, with replacement. We also did one-to-one matching with non-replacement. 

In the process, we found that one of the variables used in the original PSM exercise - 
“origl95”, whether or not the school was in the group of schools initially identified as 
qualified for the NHP - could no longer be used in the matching logistic regression, as all 
72 NHP cases were in this group. We instrumented “origl95” but the results were not 
promising: the logit correctly predicted only 2 of 72 NHP cases and the pseudo r-square for 
the stage one regression was .06. We then redid the PSM without the variable “origl95”. 
The logit correctly predicted 2 of the 72 NHP cases with a pseudo r-square of .0904. 

A literature review of recent applications of PSM to small samples yielded consensus that 
the matching should be on the basis of “replacement” - meaning that, once matched to a 
treatment school, a matched non-NHP school would be “replaced” in the pool of matched 
non-NHP schools eligible for subsequent matching to another treatment school. On the 
basis of this procedure, we identified 66 unique matched non-NHP schools matched to the 
72 treatment schools on propensity scores that differed less than .05. 

Since the NHP schools were selected across Jamaica, we added dummy variables for the 
six (one omitted) regions to the NHP prediction equation, and obtained matched non-NHP 
schools for all but one NHP school (with replacement) and 70 one-to-one matched non- 
NHP schools (without replacement). However, the logit correctly predicted only 3 of the 72 
NHP cases with a pseudo r2 of .1069 (Regression results are presented below in Table 
A.I.). Variables associated with NHP at p<.10 were shift, number of primary students (-), 
number of primary teachers, and 1999 GSAT math (-). 



^ Two schools were incorrectly identified as NHP, one school did not have a GSAT match with the School 
Census data, and one school lacked 1999 GSAT data 
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Conclusion 

Although specific criteria were given for the selection of NHP schools, empirical measures 
related to these criteria were not strongly related to NHP incidence. 



Table A1. Logistic regression, likelihood of school being selected as NHP school, Jamaica 1999. 





Odds 

Ratio 


Std. Err. 


z 


P>|z| 


[95% Conf. 


Interval] 


Remote rural location 


0.604811 


0.275006 


-1.11 


0.269 


0.248074 


1 .474545 


Rural location 


0.586479 


0.221142 


-1.42 


0.157 


0.280087 


1.228041 


Region 2 


1.752805 


0.878127 


1.12 


0.263 


0.656592 


4.679202 


Region 3 


0.831429 


0.437815 


-0.35 


0.726 


0.296209 


2.333737 


Region 4 


0.742698 


0.40079 


-0.55 


0.581 


0.257911 


2.138724 


Region 5 


0.995249 


0.463149 


-0.01 


0.992 


0.399776 


2.477688 


Region 6 


1.820952 


0.872953 


1.25 


0.211 


0.711603 


4.659713 


Board and/or Scope 


0.786287 


0.320577 


-0.59 


0.555 


0.35362 


1 .748336 


Active PTA 


2.416165 


1 .909774 


1.12 


0.264 


0.513244 


1 1 .37442 


Breakfast program 


1.239283 


0.47391 


0.56 


0.575 


0.585686 


2.622261 


Shift 


2.482168 


1.252968 


1.8 


0.072 


0.922901 


6.67586 


Enrolment in 1-6 


0.996907 


0.001545 


-2 


0.046 


0.993883 


0.999941 


Teachers in 1-6 


1.109875 


0.058616 


1.97 


0.048 


1.000736 


1.230917 


Mathematics 1999 


0.886221 


0.046916 


-2.28 


0.023 


0.798878 


0.983114 


Language 1999 


0.97441 


0.044895 


-0.56 


0.574 


0.890274 


1.066497 


Communication! 1999 


1 .594825 


0.649552 


1.15 


0.252 


0.717843 


3.543206 


Communication2 1999 


0.778728 


0.444161 


-0.44 


0.661 


0.254617 


2.381682 


Years in service at 1 6 


0.987943 


0.024733 


-0.48 


0.628 


0.940638 


1 .037627 


Years in service at school 


0.999549 


0.006535 


-0.07 


0.945 


0.986823 


1.012439 


% with CXC qualificat. 


10.03836 


18.32729 


1.26 


0.206 


0.280287 


359.5199 


% with Certificate 


8.388521 


14.19605 


1.26 


0.209 


0.304216 


231 .3074 


% with Masters 


1.197632 


0.205942 


1.05 


0.294 


0.854974 


1 .677623 



Note: 



Omitted region is 
Number of obs 
LR chi2(22) 

Prob > chi2 
Log iikeiihood 
Pseudo R2 



Brown’s Town. 

= 786 

= 51.48 

= 0.0004 

= -214.95952 

= 0.1069 
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ANNEX B. THREE NON-NHP COMPARISON GROUPS 



The present report discusses findings related to three comparison groups. This Annex 
describes these three groups, all of which are based on PSM matching exercises carried out 
in October 2005 using the logit regression presented in Annex A. 

Group March 2005: This group is the comparison group that appears in the initial 
evaluation report. When four recoding corrections were made, the logit regression that 
predicted this group could no longer be used and a new logit regression was estimated. 

New PSM scores from this new (October 2005 with dummy variables for regions) 
regression were generated for each of the 70 non-NHP schools in this group. 

Group A: PSM with replacement: This group was identified through the October 2005 
logit regression (with dummy variables for regions), in which matches were returned to the 
pool of potential comparison schools. Matches were found for all but 1 NHP school, with 
7 schools used more than once. Six of these schools were used twice and one was used 4 
times. 

Group B: PSM with non-replacement. This group was also identified through the 
October 2005 logit regression (with dummy variables for regions), but successful matches 
were not returned to the pool of potential comparison schools. Matches were found for all 
but 2 NHP schools. 

Group C: PSM with replacement and substitution. Group A is the basis for this group. 
However, selected schools in the non-NHP list were replaced by any school that was 
initially identified as a non-NHP match school in March 2005 and whose October 2005 
PSM score was the same as that of the school being replaced. A total of 40 schools from 
the March 2005 list had propensity scores within .05 of both the NHP school to which it 
was being matched and of the empirically generated match. In this case, the replacement 
was made. Interestingly, 65 percent of the Group C schools came from the initial list of 
pre-qualified schools, compared with fewer than 20 percent of either Group A or Group B 
schools. 
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Table B.1 Propensity Scores of the NHP schools and the Matching Schools from the Three 
Comparison Groups 



Pair 


NHP 


Group A 


Group B 


Group C 


March 2005 


1 


0.375424 


0.3820437 




0.411826 


0.3820437 


2 


0.35954 


0.3643285 


0.3643285 


0.33168 


0.3643285 


3 


0.244051 


0.2438581 


0.2438581 


0.243858 


0.2438581 


4 


0.286995 


0.2856985 


0.2856985 


0.285699 


0.2856985 


5 


0.343706 


0.340642 


0.340642 


0.340642 


0.340642 


6 


0.145774 


0.1451638 


0.1451638 


0.145098 


0.1451638 


7 


0.527593 


0.5162771 


0.5162771 


0.516277 


0.5162771 


8 


0.195518 


0.1961126 


0.1981155 


0.198116 


0.1961126 


9 


0.293068 


0.2961921 


0.2961921 


0.297488 


0.2961921 


10 


0.156024 


0.1570321 


0.1570321 


0.157032 


0.1570321 


11 


0.457256 


0.4554726 


0.4554726 


0.440619 


0.4554726 


12 


0.204965 


0.2047709 


0.2047709 


0.207169 


0.2047709 


13 


0.599017 










14 


0.240704 


0.2398642 


0.2398642 


0.239864 


0.2398642 


15 


0.157793 


0.157407 


0.157407 


0.157407 


0.157407 


16 


0.14722 


0.1492377 


0.1514389 


0.149238 


0.1492377 


17 


0.265 


0.2652515 


0.2652515 


0.268492 


0.2652515 


18 


0.224852 


0.2249335 


0.2249335 


0.228803 


0.2249335 


19 


0.146338 


0.1451638 


0.1511369 


0.145164 


0.1451638 


20 


0.049221 


0.049116 


0.049116 


0.048299 


0.049116 


21 


0.152299 


0.1520756 


0.1520756 


0.152076 


0.1520756 


22 


0.126812 


0.1272792 


0.1272792 


0.125175 


0.1272792 


23 


0.14602 


0.1451638 


0.1492377 


0.145164 


0.1451638 


24 


0.037552 


0.0373183 


0.0373183 


0.039193 


0.0373183 


25 


0.119864 


0.1198781 


0.1202041 


0.120953 


0.1198781 


26 


0.366163 


0.3643285 


0.4116485 


0.364329 


0.3643285 


27 


0.017604 


0.0175418 


0.0175418 


0.02134 


0.0175418 


28 


0.052105 


0.0521836 


0.0521836 


0.052184 


0.0521836 


29 


0.182433 


0.182601 


0.182601 


0.182601 


0.182601 


30 


0.137381 


0.1378225 


0.1378225 


0.137865 


0.1378225 


31 


0.171599 


0.1708269 


0.1708269 


0.175833 


0.1708269 


32 


0.370348 


0.3643285 


0.4118264 


0.323061 


0.3643285 


33 


0.087431 


0.0874422 


0.088293 


0.089899 


0.0874422 


34 


0.085475 


0.0855861 


0.0855861 


0.085359 


0.0855861 


35 


0.097525 


0.0978607 


0.0978607 


0.097861 


0.0978607 


36 


0.140917 


0.1401056 


0.1418864 


0.140106 


0.1401056 


37 


0.098196 


0.0981866 


0.0981866 


0.098187 


0.0981866 


38 


0.036256 


0.0361684 


0.0361684 


0.037038 


0.0361684 


39 


0.030572 


0.0305642 


0.0305642 


0.031783 


0.0305642 


40 


0.087294 


0.0874422 


0.0874422 


0.088522 


0.0874422 


41 


0.296246 


0.2961921 


0.2974878 


0.323061 


0.2961921 


42 


0.023784 


0.0238217 


0.0238217 


0.023822 


0.0238217 


43 


0.069631 


0.0697395 


0.0697395 


0.067822 


0.0697395 


44 


0.132974 


0.1330859 


0.1330859 


0.132622 


0.1330859 


45 


0.02821 


0.0282001 


0.0282001 


0.0282 


0.0282001 


46 


0.0457 


0.0456606 


0.0456606 


0.044159 


0.0456606 


47 


0.048523 


0.0484426 


0.0484426 


0.047609 


0.0484426 


48 


0.095819 


0.0957442 


0.0957442 


0.095744 


0.0957442 


49 


0.121404 


0.1214606 


0.1214606 


0.121788 


0.1214606 


50 


0.062639 


0.0627135 


0.0627135 


0.062714 


0.0627135 


51 


0.162218 


0.1625419 


0.1625419 


0.162542 


0.1625419 


52 


0.104255 


0.1043026 


0.1043026 


0.103187 


0.1043026 


53 


0.010005 


0.0093991 


0.0093991 


0.009399 


0.0093991 


54 


0.163886 


0.16362 


0.16362 


0.168216 


0.16362 


55 


0.094261 


0.0942395 


0.0942395 


0.09424 


0.0942395 


56 


0.101796 


0.1017858 


0.1017858 


0.101786 


0.1017858 


57 


0.085549 


0.0855861 


0.085767 


0.086832 


0.0855861 


58 


0.082087 


0.0819854 


0.0819854 


0.081985 


0.0819854 


59 


0.14004 


0.1399766 


0.1399766 


0.139977 


0.1399766 


60 


0.119571 


0.1198781 


0.1198781 


0.120204 


0.1198781 


61 


0.36321 1 


0.3643285 


0.3820437 


0.364329 


0.3643285 


62 


0.148985 


0.1492377 


0.1517204 


0.149238 


0.1492377 



61 
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Pair 


NHP 


Group A 


Group B 


Group C 


March 2005 


63 


0.140839 


0.1401056 


0.1401056 


0.140106 


0.1401056 


64 


0.061841 


0.0619062 


0.0619062 


0.061374 


0.0619062 


65 


0.196829 


0.1961126 


0.2006823 


0.202354 


0.1961126 


66 


0.069026 


0.0690925 


0.0690925 


0.066765 


0.0690925 


67 


0.193181 


0.1919115 


0.1961126 


0.191912 


0.1919115 


68 


0.094822 


0.0947867 


0.0947867 


0.094787 


0.0947867 


69 


0.072284 


0.0724146 


0.0724146 


0.072955 


0.0724146 


70 


0.191439 


0.1919115 


0.1919115 


0.191912 


0.1919115 


71 


0.12774 


0.1275111 


0.1275111 


0.125659 


0.1275111 


72 


0.278472 


0.2849856 


0.2849856 


0.284986 


0.2849856 



62 
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Jamaica AED School Survey 2005 
FOR PRINCIPAL AND TEACHERS 



The Jamaica AED School Survey (JASS) was administered in October and 
November of 2005 to 139 NHP schools and their matched pairs by the Caribbean 
Consulting Group (CCG). CCG completed this work under a contact with The 
Academy for Educational Development (AED), 



The survey consists of 83 questions. In all, 246 discrete data elements were 
collected from these 83 questions. 

The survey is divided into twelve sections covering questions on the following 
topics: 



SECTION I: 
SECTION II: 
SECTION III: 
SECTION IV. 
SECTION V: 
SECTION VI: 
SECTION VII: 
SECTION VIII. 
SECTION IX. 
SECTION X. 
SECTION XI. 
SECTION XII 



SCHOOL INFORMATION 

SCHOOL IMPROVEMENT PLANS AND RESOURCES 

INTRODUCTION OF THE REVISED PRIMARY CURRICULUM 

SUPPLEMENTARY MATERIALS 

PRINCIPAL TRAINING 

RESOURCE TEACHER TRAINING 

SCHOOL-BASED TRAINING BY OUTSIDE SPECIALISTS 

TRAINING PROVIDED BY SUPPORT TEACHERS 

COMMUNITY AND PARENT TRAINING 

HEALTH AND NUTRITION 

COMPUTER USE IN THE SCHOOL 

GRADE 4 LITERACY 
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Jamaica AED School Survey 2005 
INTERVIEWER QUESTIONNAIRE 



Interviewer Name: 


Date of interview: Month 


Dav 


2005 


Time interview started: 






Time interview ended: 







Directions for selecting respondents: Choose three respondents: the 
principal, one teacher of grades 1-3 and one teacher of grades 4-6. 

Choose resource teachers/ coordinators if they are available. Conduct the 
interview with this GROUP, not individually. The teachers should have 
worked in the school for at least the past 3 years. Get information about 
what year each of the respondents started working at this school. 



SECTION I: SCHOOL INFORMATION 

Ql. School Name (in block letters) 

Q2. School ID (from list provided) Place on top of every page 

Q3 School address 

Q4. School telephone number 

Q5. School email address 

Q6. School type Circle ONE only 

(a) primary, (b) primary and infant, (c) all age, (d) primary and junior high, (e) other 



Jamaica MOEYC, USAID and Academy for Educational Development School Survey. 
October 2005. All rights reserved. Do not cite, duplicate or distribute. 



Jamaica AED School Survey 
ID Page 3 





School Type 




NHP 


Comp 


Missing 


4.2% 


3.0% 


Ali Age 


37.5% 


29.9% 


Other 


2.8% 


6.0% 


Primary 


43.1% 


49.3% 


Primary & infant 


6.9% 


6.0% 


Primary & Junior High 


5.6% 


6.0% 


Totai 


100.0% 


100.0% 



Name of Principal 


07. In school since 


Name of Teacher for Grades 1-3 


08 In school since 


Name of Teacher for Grades 4-6 


09. In school since 





Nh 

NHP 


HP 

Comp 


Mean 


Mean 


Principai_in_school_since 


1986 


1987 


Teacher #1 ln_schooi_since 


1990 


1991 


Teacher #2 in_school_since 


1990 


1993 



QIO. How might other schools have heard about this school? Has it been designated with 
a special term, such as a Professional Development Center, an IT Pilot school, a “best 
practice” for teaching and learning Demonstration School? CHECK ALL that 
respondents mention but DO NOT READ list 

Professional Development Center 

Demonstration School (“best practice”) 

Learning Center 

Lighthouse School 

Pilot School for the Revised Primary Curriculum 

Pilot School for Instructional Technology (IT) 

Adult Literacy Center 

Other 

None 
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# of Designations 


School Type 




NHP 


Comp 


0 


13.9% 


49.3% 


1 


54.2% 


46.3% 


2 


20.8% 


1 .5% 


3 


5.6% 


1 .5% 


5 


1 .4% 




Missing 


4.2% 


1 .5% 


Total 


100.0% 


100.0% 



SECTION II: SCHOOL IMPROVEMENT PLANS AND 

RESOURCES 



SAY: School Improvement Plans (or School Development Plans) are 
documents that schools develop; they analyze the problems of the school 
and recommend solutions. In some cases, a special group (the School 
Improvement Action Group) works on these plans. In some cases, schools 
prepare Annual Action Plans that specify what the school will do, over a 3 
or 5-year period. And some of these School Improvement Plans focus on 
improving students’ reading and math skills. 

The Government of Jamaica and donors to education have provided 
support to primary education in Jamaica. This support includes many 
different types of inputs to schools. Not all schools have received the 
same inputs. Some schools have received small cash grants, while other 
schools have received resources from the Ministry of Education, Youth 
and Culture (MOEYC) or donations from local sponsors. This section asks 
about School Improvement Plans and donations received by the school to 
support these plans. 



Q1 1. Does this school have a School Improvement Action group that was trained to 
analyze school problems and contribute to school decision-making? Circle ONE only 

(a) Yes, there is a trained group 

(b) Yes, there is a group but it has not been trained 

(c) No 





School Type 




NHP 


Comp 


Missing 


4.2% 


4.5% 


No 


6.9% 


17.9% 


Yes but not trained 


37.5% 


52.2% 


Yes Trained Group 


51 .4% 


25.4% 


Total 


100.0% 


100.0% 
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Q12. Did this school develop a School Improvement Plan or School Development Plan 
for the purpose of improving the reading and math skills of students at the school? 

Circle ONE only 

(a) Yes, the school has a School Improvement Plan that includes math and reading 

(b) Yes, the school has a SIP/SDP, but it does not include math and reading 

(c) Yes, the school is in the process of developing a School Improvement Plan 

(d) No If No, Skip to Q 1 5 





School Type 


Total 


NHP 


Comp 


Missing 


4.2% 


1 .5% 


2.9% 


No 


1 .4% 


3.0% 


2.2% 


Yes in Process of SiP 




4.5% 


2.2% 


Yes Schooi has a SIP 


94.4% 


85.1% 


89.9% 


Yes SIP but not Math & reading 




6.0% 


2.9% 


Total 


100.0% 


100.0% 


100.0% 



Q13. Does the School Improvement Plan include a school-wide Annual Action Plan for 
improving the reading and math skills of students at the school? Circle ONE only 

(a) Yes for a 3-year plan period (give dates) 

(b) Yes for a 5-year plan period (give dates) 

(c) No If No, Skip to Q1 5 





School Type 




NHP 


Comp 


Total 


Missing 


6.9% 


4.5% 


5.8% 


1 year plan 


1 .4% 




.7% 


No 


1 .4% 


9.0% 


5.0% 


Yes 3 Yr Plan 


63.9% 


61 .2% 


62.6% 


Yes 5 Yr Plan 


26.4% 


25.4% 


25.9% 


Total 


100.0% 


100.0% 


100.0% 



Q14. Who developed the Annual Action Plan? That is, who were the members of the 
School Improvement Plan/School Development Plan Action Group? CHECK ALL that 
respondents mention but DO NOT READ list 

Board Chairman 

Other Board members 

Parent representatives from the PTA 

Other parents 

Community members not from PTA 
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Principal 

Teachers 

Former students (alumni) 
Other 



# of Groups 
mentioned 


Schooi Type 


Totai 


NHP 


Comp 


0 


2.8% 


10.4% 


6.5% 


1 


2.8% 


4.5% 


3.6% 


2 


22.2% 


22.4% 


22.3% 


3 


18.1% 


10.4% 


14.4% 


4 


19.4% 


17.9% 


18.7% 


5 


12.5% 


16.4% 


14.4% 


6 


11.1% 


13.4% 


12.2% 


7 


5.6% 


3.0% 


4.3% 


8 


1 .4% 




.7% 


Missing 


4.2% 


1 .5% 


2.9% 


Totai 


1 00.0% 


100.0% 


100.0% 



Q 15. In the past 5 years, has any local sponsor (such as Kiwanis, Rotary, Lion’s Club, a 
local church. Adopt- a- School) helped with resources (cash or in-kind), either regularly or 
through a special project, for any purpose? Circle ONE only 

(a) Yes. Who is the sponsor? 

(b) No 





Schooi Type 




NHP 


Comp 


No 


17.4% 


24.2% 


Yes 


82.6% 


75.8% 


Totai 


100.0% 


100.0% 



Q16. In the past 5 years, has the school received additional resources (from any source) 
for the purpose of improving students’ reading and math skills? By additional resources, 
we mean resources that are not those regularly provided by the MOEYC. Circle ONE 
only 

(a) Yes 

(b) No If No, Skip to Q 1 9 





Schooi Type 


Total 


NHP 


Comp 


No 


4.3% 


21 .2% 


12.6% 


Yes 


95.7% 


78.8% 


87.4% 


Totai 


100.0% 


100.0% 


100.0% 
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Q17. What sort of additional resources has this school received over the past 5 years? 
Please look over the following list, and indicate all the types of additional resources this 
school has received. ? CHECK ALL that respondents mention. 

In-kind donations of hard goods for instructional purposes, such as computers 

or books (Skip to Q1 9) 

In-kind donations of hard goods to improve school facility, such as adding 

fences, repairing desks, paving driveway (Skip to Q19) 

Technical assistance, such as teacher training (Skip to Q19) 

Funds raised for special instructional projects (Ask Q18) 

Funds raised for special non-instructional projects, such as for school meals 

(AskQ18) 

Other actual cash donations (AskQ18) 



Number indicated 


School Type 


Total 


NHP 


Comp 


0 


4.2% 


20.9% 


12.2% 


1 


6.9% 


13.4% 


10.1% 


2 


20.8% 


16.4% 


18.7% 


3 


20.8% 


22.4% 


21 .6% 


4 


19.4% 


14.9% 


17.3% 


5 


16.7% 


7.5% 


12.2% 


6 


6.9% 


3.0% 


5.0% 


Missing 


4.2% 


1 .5% 


2.9% 


Total 


100.0% 


100.0% 


100.0% 



Q18. Who provided these funds or cash donations? CHECK ALL that respondents 
mention but DO NOT READ list 

GOJ/MOEYC (in addition to regular grant) 

A local charitable foundation or sponsor 

Teachers, through their fund raising efforts 

The school’s PTA 

Former students of the school (alumnae) 

The local community or parents of students 

A local company 

A local politician/ member of Parliament 

Adopt- a- school 

Tuck shop 

Other 
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Number inidcated 


School Type 


Total 


NHP 


Comp 


0 


29.2% 


40.3% 


34.5% 


1 


15.3% 


19.4% 


17.3% 


2 


22.2% 


20.9% 


21 .6% 


3 


11.1% 


1 1 .9% 


1 1 .5% 


4 


12.5% 


6.0% 


9.4% 


5 


2.8% 




1 .4% 


6 


1 .4% 




.7% 


8 


1 .4% 




.7% 


Missing 


4.2% 


1 .5% 


2.9% 


Total 


100.0% 


100.0% 


100.0% 



Q19. Did any of the resources that the school received directly support a School 
Improvement or School Development Plan to improve students’ reading and math skills? 

Circle ONE only. 

(a) Yes 

(b) No 

(c) This school does not have a School Improvement or School Development Plan 





School Type 






NHP 


Comp 


Total 


Missing 


5.6% 


4.5% 


5.0% 


No 


1 .4% 


1 9.4% 


10.1% 


School Does not 
have an SIP 




3.0% 


1 .4% 


Yes 


93.1% 


73.1% 


83.5% 


Total 


1 00.0% 


100.0% 


100.0% 
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SECTION III: INTRODUCTION OF THE REVISED 
PRIMARY CURRICULUM 



SAY: I’m now going to ask some questions about the Revised Primary 
Curriculum. 



Q20. In some schools the Revised Primary Curriculum was introduced all at once, and in 
other schools it was introduced over several years, by grade. Please look at the following 
options and choose the one that most closely describes how the revised primary 
curriculum was introduced in this school. ? Circle ONE only. 



(a) Introduced in all grades during school years 1997/98 and 1998/99 

(b) Introduced in all grades during school year 1999/2000 

(c) Phased introduction: 

School year 2000/2001 for Grades 1 and 4 
School year 2001/2002 for Grades 2 and 5 
School year 2002/2003 for Grades 3 and 6 

(d) Introduced, but in a different way than above (e.g. the school obtained a 
photocopy of the revised curriculum before it was formally introduced) 

(e) Not yet introduced 





School Type 




NHP 


Comp 


Missing 


5.6% 


1 .5% 


Introduced but in a 
different way 


6.9% 


4.5% 


Introduced in all grades 
during 1999/2000. 


25.0% 


1 .5% 


Introduced in all grades 
during 97/98 & 98/99 


54.2% 




Not yet introduced 


1 .4% 




Phased introduction 


6.9% 


92.5% 


Total 


100.0% 


100.0% 



Q21. We are aware that teachers received special training when the revised primary 
curriculum was introduced. Where did the training of teachers in this school take place, 
and who conducted the training? Please look at the following list and tell me all the ways 
teachers in this school were trained for the Revised Primary Curriculum. CHECK ALL 
that respondents mention 

Initial 4-day training workshops, by grade, in large centers 

Cluster training workshops with teachers from other schools 

Workshops in the school, given by education officers 

Other. Please describe 
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Number of selections 


School Type 


Total 


NHP 


Comp 


0 


1 .4% 




.7% 


1 


9.7% 


22.4% 


15.8% 


2 


36.1% 


46.3% 


41 .0% 


3 


36.1% 


23.9% 


30.2% 


4 


12.5% 


6.0% 


9.4% 


Missing 


4.2% 


1 .5% 


2.9% 


Total 


100.0% 


100.0% 


100.0% 



SAY: The next two questions ask you to estimate the amount of training in 
the revised primary curriculum that teachers in this school have received. 
This training includes summer training in large groups, cluster training 
during the school year, training in the school during the school year and 
any other type of training related to the revised primary curriculum 
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Q22. About what percent of the current teachers in Grades 1-3 have received more than 
10 days of training related to the revised curriculum since it was introduced in this 
school? percent 





School Type 


Total 


NHP 


Comp 


0 


4.4% 


38.5% 


21.1% 


5 




1 .5% 


.8% 


9 




1 .5% 


.8% 


10 




1 .5% 


.8% 


16 


1 .5% 




.8% 


17 


1 .5% 




.8% 


25 


1 .5% 


3.1% 


2.3% 


33 


7.4% 


9.2% 


8.3% 


40 


1 .5% 




.8% 


44 


1 .5% 


1 .5% 


1 .5% 


50 


16.2% 


12.3% 


14.3% 


57 




3.1% 


1 .5% 


60 




1 .5% 


.8% 


66 


7.4% 


1 .5% 


4.5% 


67 


5.9% 


3.1% 


4.5% 


70 


1 .5% 


1 .5% 


1 .5% 


71 


2.9% 




1 .5% 


75 


2.9% 


3.1% 


3.0% 


78 


1 .5% 




.8% 


80 


1 .5% 


1 .5% 


1 .5% 


83 


7.4% 




3.8% 


85 


1 .5% 




.8% 


90 


2.9% 




1 .5% 


94 


1 .5% 




.8% 


100 


27.9% 


15.4% 


21 .8% 


Total 


100.0% 


100.0% 


100.0% 





School Type 


NHP 


Comp 


Mean 


Mean 




68 


37 
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Q23. About what percent of the current teachers in Grades 4-6 have received more than 
10 days of training related to the revised curriculum since it was introduced in this 
school? percent 





School Type 


Total 


NHP 


Comp 


0 


5.9% 


50.0% 


27.6% 


16 


1 .5% 




.7% 


20 




1 .5% 


.7% 


25 


1 .5% 


1 .5% 


1 .5% 


30 


1 .5% 




.7% 


33 


4.4% 


3.0% 


3.7% 


40 




1 .5% 


.7% 


42 


1 .5% 




.7% 


43 


1 .5% 




.7% 


50 


17.6% 


7.6% 


12.7% 


56 


1 .5% 




.7% 


60 


1 .5% 




.7% 


66 


2.9% 


1 .5% 


2.2% 


67 


4.4% 


4.5% 


4.5% 


71 




1 .5% 


.7% 


75 


1 .5% 


1 .5% 


1 .5% 


80 




4.5% 


2.2% 


81 


1 .5% 




.7% 


83 


2.9% 




1 .5% 


85 


2.9% 




1 .5% 


86 


1 .5% 


1 .5% 


1 .5% 


89 


1 .5% 




.7% 


90 




3.0% 


1 .5% 


92 


2.9% 




1 .5% 


100 


39.7% 


16.7% 


28.4% 


Total 


100.0% 


100.0% 


100.0% 





School Type 


NHP 


Comp 


Mean 


Mean 




71 


37 
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SAY: Some schools have had special activities that support the revised 
primary curriculum. The next questions ask about these activities. 



Q24. Does this school have a Drop Everything and Read program? Circle ONE only. 

(a) Yes, for about 15 minutes per day 

(b) Yes, for more than 15 minutes per day 

(c) No 





Schooi Type 


Totai 


NHP 


Comp 


Missing 


5.6% 


4.5% 


5.0% 


No 


20.8% 


29.9% 


25.2% 


Yes about 15 Min/day 


40.3% 


44.8% 


42.4% 


Yes more than 15 Min/day 


33.3% 


20.9% 


27.3% 


Totai 


1 00.0% 


1 00.0% 


1 00.0% 



Q25. Does this school have a Family Literacy (such as Reading Starts with Us) program? 

Circle ONE only. 

(a) Yes 

(b) No 





Schooi Type 






NHP 


Comp 


Totai 


no 


92.8% 


89.4% 


91.1% 


yes 


7.2% 


10.6% 


8.9% 


Totai 


100.0% 


100.0% 


100.0% 



Q26. Does this school have afternoon or evening programs for parents to help them 
understand the Revised Primary Curriculum so they can help their children? Circle 
ONE only. 

(a) Yes 

(b) No 





Schooi Type 


Totai 


NHP 


Comp 


No 


91 .3% 


83.3% 


87.4% 


Yes 


8.7% 


16.7% 


12.6% 


Totai 


100.0% 


100.0% 


100.0% 
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Q27. Does this school have a Competency Shelter? Circle ONE only. 

(a) Yes. Please describe what it is: 

(b) No 





School Type 


Total 


NHP 


Comp 


No 


75.4% 


84.8% 


80.0% 


Yes 


24.6% 


15.2% 


20.0% 


Total 


100.0% 


100.0% 


100.0% 



Q28. Has this school been the venue for a Literacy Summer Camp? Circle ONE only. 

(a) Yes, for five (5) or more summers 

(b) Yes, for two (2) to four (4) summers 

(c) Yes, for one summer 

(d) No, this school has never been the venue of a Literacy Summer Camp 





School Type 


Total 


NHP 


Comp 


Missing 


5.6% 


4.5% 


5.0% 


No 


8.3% 


14.9% 


1 1 .5% 


Yes 2 to 4 Summers 


31.9% 


35.8% 


33.8% 


Yes 5 or more summers 


47.2% 


41 .8% 


44.6% 


Yes one summer 


6.9% 


3.0% 


5.0% 


Total 


100.0% 


100.0% 


1 00.0% 



Q29. Are there any other special activities or programs that this school has implemented 
in support of the Revised Primary Curriculum? Circle ONE only. 

(a) Yes. Please describe: 

(b) No 





School Type 


Total 


NHP 


Comp 


No 


15.9% 


36.4% 


25.9% 


Yes 


84.1% 


63.6% 


74.1% 


Total 


100.0% 


100.0% 


100.0% 
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SECTION IV. SUPPLEMENTARY MATERIALS 



SAY: This section refers to materials that are not regularly provided to the 
school by the MOEYC, but are in addition to the regular materials. We 
refer to these as “supplementary” materials. We are interested in 
supplementary materials that support the Revised Primary Curriculum. 



Q30. During the past 5 years, has this school received any supplementary reading 
materials for literacy? Circle ONE only. 

(a) Yes 

(b) No If no, skip to Q32 





School Type 


Total 


NHP 


Comp 


No 


7.2% 


21 .2% 


14.1% 


Yes 


92.8% 


78.8% 


85.9% 


Total 


100.0% 


100.0% 


100.0% 



Q31. What sorts of supplementary reading materials did this school receive? CHECK 

ALL that respondents mention but DO NOT READ list 

Textbooks related to the curriculum 

“Big books” for shared reading 

Story books for the teacher to read to children 

A set of identical student books for shared reading 

Books for student independent reading 

Other 



Number of 
responses 


School Type 


Total 


NHP 


Comp 


0 


7.2% 


21 .2% 


14.1% 


1 


1 1 .6% 


12.1% 


1 1 .9% 


2 


27.5% 


31 .8% 


29.6% 


3 


34.8% 


24.2% 


29.6% 


4 


14.5% 


10.6% 


12.6% 


5 


4.3% 




2.2% 


Total 


100.0% 


100.0% 


100.0% 
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Q32. How many classrooms have reading corners or classroom libraries (shelves) 





School Type 


Total 


NHP 


Comp 


0 


1 .5% 


3.0% 


2.2% 


2 


1 .5% 




.7% 


3 


7.4% 


12.1% 


9.7% 


4 


4.4% 


16.7% 


10.4% 


5 


4.4% 


3.0% 


3.7% 


6 


19.1% 


22.7% 


20.9% 


7 


2.9% 


4.5% 


3.7% 


8 


5.9% 


3.0% 


4.5% 


9 


2.9% 


4.5% 


3.7% 


10 


1 .5% 


3.0% 


2.2% 


11 




1 .5% 


.7% 


12 


4.4% 


1 .5% 


3.0% 


13 


2.9% 




1 .5% 


14 


4.4% 


1 .5% 


3.0% 


15 


4.4% 




2.2% 


16 


2.9% 




1 .5% 


18 


2.9% 


4.5% 


3.7% 


19 




1 .5% 


.7% 


20 


1 .5% 




.7% 


21 


1 .5% 


1 .5% 


1 .5% 


24 


2.9% 




1 .5% 


25 




1 .5% 


.7% 


27 




1 .5% 


.7% 


30 


1 .5% 




.7% 


32 


1 .5% 




.7% 


37 


1 .5% 




.7% 


100 


16.2% 


12.1% 


14.2% 


Total 


100.0% 


100.0% 


100.0% 





NHP 


NHP 


Comp 


Mean 


Mean 




25 


19 
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Q33. Is there a sehool library? Circle ONE only. 

(a) Yes 

(b) No 





School Type 


Total 


NHP 


Comp 


No 


13.0% 


13.6% 


13.3% 


Yes 


87.0% 


86.4% 


86.7% 


Total 


100.0% 


100.0% 


100.0% 



Q34. During the past 5 years, has this sehool received any supplementary mathematics 
materials? Circle ONE only. 

(a) Yes 

(b) No If No, Skip to Section V 





School Type 


Total 


NHP 


Comp 


No 


1 1 .6% 


45.5% 


28.1% 


Yes 


88.4% 


54.5% 


71 .9% 


Total 


100.0% 


100.0% 


100.0% 
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Q35. What sorts of supplementary math materials did this school receive? CHECK 
ALL that respondents mention but DO NOT READ list 

Textbooks related to the curriculum 

Math activity books 

Graph books for students 

Sets of teacher guides (e.g. Nelson Primary Maths for Caribbean Schools) 

Mathematics software on CDs (e.g. Math Workshop Deluxe) 

Professional periodicals for teachers (e.g. NCTM’s Teaching Children Math) 

Calculators for students to use 

Manipulatives such as blocks, rods, geometric shapes 

Measurement tools such as rulers, protractors, compasses 

Other 



Number of 
Responses 


School Type 




NHP 


Comp 


Total 


Q35n 0 


10.1% 


43.9% 


26.7% 


1 




12.1% 


5.9% 


2 


10.1% 


13.6% 


1 1 .9% 


3 


15.9% 


18.2% 


17.0% 


4 


18.8% 


7.6% 


13.3% 


5 


15.9% 


3.0% 


9.6% 


6 


13.0% 


1 .5% 


7.4% 


7 


2.9% 




1 .5% 


8 


4.3% 




2.2% 


9 


4.3% 




2.2% 


10 


4.3% 




2.2% 


Total 


100.0% 


100.0% 


100.0% 



Jamaica MOEYC, USAID and Academy for Educational Development School Survey. 
October 2005. All rights reserved. Do not cite, duplicate or distribute. 



Jamaica AED School Survey 
ID Page 19 



Section V: PRINCIPAL TRAINING 



SAY: The next questions ask about the person who was the principal of the 
school during the school year 2003-2004. This could be the current 
principal, or it could be someone who is no longer at this school. 



Q36. Did the principal who served during 2003-2004 participate in specialized training 
for principals at any time since the school year 2000? Circle ONE only. 

(a) Yes 

(b) No If No, skip to Section VI 

(c) Don’t know If Don’t know, skip to Section VI 





Schooi Type 


Totai 


NHP 


Comp 


Missing 


6.9% 


3.0% 


5.0% 


Don't Know 


1 .4% 


3.0% 


2.2% 


No 


1 .4% 


9.0% 


5.0% 


Yes 


90.3% 


85.1% 


87.8% 


Totai 


1 00.0% 


100.0% 


100.0% 
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Q37. What type of training did the principal who served the school during the school year 
2003-2004 participate in? We realize that not all principals participated in all types of 
training, and we are interested only in the types of training received by the principal of 
this school. Please look over the list of possible training opportunities, and tell me which 
ones the principal of this school participated in. CHECK ALL that respondents 
mention 

Overseas summer training at Mt. St. Vincent University in Canada 

Training provided by Mt. St. Vincent University in Jamaica (at Bethlehem, 

Case, Sam Sharpe, or St. Joseph Teachers College) 

Training in School Improvement Planning 

Workshop on community mobili z ation 
Workshop on parent participation facilitation skills 

Training in leadership and management skills for peer mentoring 

Peer learning with other principals (e.g. cluster school meetings, NHP 

principal’s network) 

Performance management training 

Implementing the revised primary curriculum 

Beginning principal training 

Training occurred, but not sure of content 

Other. Please describe 





NHP 


NHP 


Comp 


Sum 


Sum 


Overseas Summer Training 


13 


9 


Training MtStVincent 


29 


31 


Training in SIP 


55 


36 


Workshop Community 


21 


9 


Workshop Parent 


28 


9 


Training Leadership 


33 


22 


Peer Learning 


51 


29 


Performance 


38 


22 


Impiementing 


55 


39 


Principai Training 


20 


19 


Training Occurred 


9 


9 


Other 


10 


7 
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SECTION VI: RESOURCE TEACHER TRAINING 



SAY: In some primary schools, experienced teachers have been identified 
as persons who can provide support to other teachers. These support 
teachers are called Resource Teachers or Literacy/Numeracy/ 
Assessment Coordinators. Some larger schools have Support Teachers 
that help other teachers on a grade by grade basis, called Grade 
Coordinators. In the next section we call all of these types of teachers 
Support Teachers. In some cases, these support teachers have received 
specialized training. This section asks about these school-based support 
teachers and the types of training they have received. 



Q38. First, in the last school year, 2004-2005, did you have any Support Teachers for 
Grades 1-6? Circle ONE only. 

(a) Yes. How many? teachers 

(b) No. If No, Skip to Section VII 





School Type 


Total 


NHP 


Comp 


No 


14.5% 


33.3% 


23.7% 


Yes 


85.5% 


66.7% 


76.3% 


Total 


100.0% 


100.0% 


100.0% 
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Q39. What types of special training did these Support Teachers receive over the past 5 
years to help them provide assistance to other teachers in this school? Please review the 
list of topics for courses or workshops for Support Teachers, and tell me any that Support 
Teachers in this school participated in. Check ALL that respondents mention. 

How to teach other teachers (e.g. how to teach adult peers e.g.) 

How to do peer to peer observation (e.g. Reciprocal Teaching Technique) 

How to teach the Revised Curriculum for grades 1-6 

The integrated approach to teaching grades 1-3 

How to carry out Continuous Assessment 

How to teach Cooperative Learning 

Visualization in Participative Programs (VIPP) Methodology 

Communication/Listening 

Motivation 

None 





NHP 


NHP 


Comp 


Yes 


Yes 


Teach teachers 


36 


13 


Peer to peer 


40 


10 


Teach Revised 


54 


31 


Integrated 


59 


27 


Continuous 


59 


42 


Cooperative 


59 


24 


Visuaiization 


31 


7 


Communication 


34 


10 


Motivation 


35 


12 


none 


0 


1 
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Q40. During the last school year, 2004-2005 including the summer, about how many 
days of training did each Support Teacher receive? Answer only for Support 
Teachers for Grades 1-6 who were in this school during 2004-2005. 



Math Support Teacher days 

Reading Support Teacher days 

Assessment Coordinator days 

Other Support Teacher days 



Math Support 
Teacher 


School Type 




NHP 


Comp 


Total 


0 


12.2% 


38.5% 


18.5% 


1 


2.4% 


7.7% 


3.7% 


2 


7.3% 


7.7% 


7.4% 


3 


9.8% 


15.4% 


11.1% 


4 


9.8% 


7.7% 


9.3% 


5 


34.1% 


15.4% 


29.6% 


6 


2.4% 




1 .9% 


8 


4.9% 




3.7% 


10 


7.3% 


7.7% 


7.4% 


15 


4.9% 




3.7% 


25 


2.4% 




1 .9% 


35 


2.4% 




1 .9% 


Total 


100.0% 


100.0% 


100.0% 



Reading Support 


School Type 


Total 


NHP 


Comp 


0 


6.8% 


31 .3% 


13.3% 


1 


2.3% 


6.3% 


3.3% 


2 


6.8% 


31.3% 


13.3% 


3 


20.5% 


6.3% 


16.7% 


4 


9.1% 


6.3% 


8.3% 


5 


25.0% 


6.3% 


20.0% 


6 


2.3% 




1 .7% 


8 


6.8% 




5.0% 


10 


1 1 .4% 


6.3% 


10.0% 


15 


6.8% 




5.0% 


16 




6.3% 


1 .7% 


35 


2.3% 




1 .7% 


Total 


100.0% 


100.0% 


100.0% 
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Assessment 

Coordinator 


School Type 




NHP 


Comp 


Total 


0 




13.8% 


6.2% 


1 




17.2% 


7.7% 


2 


19.4% 


10.3% 


15.4% 


3 


25.0% 


10.3% 


18.5% 


4 


13.9% 


20.7% 


16.9% 


5 


25.0% 


10.3% 


18.5% 


6 


5.6% 


3.4% 


4.6% 


8 




3.4% 


1 .5% 


10 


8.3% 


10.3% 


9.2% 


15 


2.8% 




1 .5% 


Total 


100.0% 


100.0% 


1 00.0% 



Other Support 


School Type 


Total 


NHP 


Comp 


0 


14.3% 


37.5% 


24.3% 


1 


4.8% 


12.5% 


8.1% 


2 


4.8% 


12.5% 


8.1% 


3 


9.5% 


12.5% 


10.8% 


4 


9.5% 


12.5% 


10.8% 


5 


33.3% 




18.9% 


6 




6.3% 


2.7% 


8 


9.5% 




5.4% 


10 


9.5% 


6.3% 


8.1% 


15 


4.8% 




2.7% 


Total 


100.0% 


100.0% 


100.0% 



SECTION VII: SCHOOL-BASED TRAINING BY OUTSIDE 

SPECIALISTS 



SAY: Several different types of in-service teacher training have been 
offered to teachers of primary grades in Jamaica. This question refers to 
training that takes place in the school, or school-based training. In-service 
training can include supervised practice and feedback, demonstrations, 
workshops and much more. We are asking about training that was 
provided by specialists who came from outside the school, such as 
regional or district office teacher trainers, project teacher trainers, 
resource teachers from other schools or other specialists who were not 
directly teaching at this school. For these questions, we are asking about 
training opportunities for regular classroom teachers, not for Support 
Teachers. 
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Q41. In the past 5 years, have regular teachers of Grades 1-6 in this school participated in 
any school-based inservice teacher training provided by outside specialists? Circle 
ONE only. 

(a) Yes 

(b) No. If no, skip to Section VIII 





School Type 


Total 


NHP 


Comp 


No 


10.1% 


39.4% 


24.4% 


Yes 


89.9% 


60.6% 


75.6% 


Total 


100.0% 


100.0% 


100.0% 



Q42. What types of school-based inservice training for mathematics in Grades 1-6 have 
outside specialists provided to regular teachers in this school? Please review the list of 
types of training and indicate any that teachers in Grades 1-6 have received. Check ALL 
that respondents mention. 

Clinical Supervisory Practice (specialist consults with teacher, observes 

teaching and provides verbal/written/printed feedback) 

Demonstration of teaching strategies by mathematics specialist 

Workshops conducted by mathematics specialist 

Unannounced observation of teaching and feedback by MOEYC education 

officer (territorial, district or regional) 

Continuous Assessment 

Cooperative Learning 

Other 

None for primary teachers 





NHP 


NHP 


Comp 


Yes 


Tes 


Clinical Supervisory 


48 


3 


Demonstration 


56 


8 


Workshops 


46 


14 


Observations 


50 


14 


Continuous 


40 


16 


Cooperative 


36 


11 


Other 


6 


2 


None 


2 


10 



Q43. What types of school-based inservice training for literacy in Grades 1-6 have 
outside specialists provided to regular teachers in this school? Please review the list of 
types of training and indicate any that teachers in Grades 1-6 have received. Check ALL 
that respondents mention. 

Clinical Supervisory Practice (specialist consults with teacher, observes 

teaching and provides verbal/written/printed feedback) 
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Demonstration of teaching strategies by literacy specialist 

Workshops conducted by literacy specialist 

Unannounced observation of teaching and feedback by MOEYC education 

officer (territorial, district or regional) 

Continuous Assessment 

Cooperative Learning 

Other 

None for primary teachers 





NHP 




NHP 


Comp 




Sum 


Sum 


Clinical Supervisory 


50 


7 


Demonstration 


51 


22 


Workshops 


39 


24 


Observations 


48 


16 


Continuous Assessment 


37 


18 


Cooperative 


40 


12 


other 


5 


4 


None 


2 


10 



SECTION VIII: TRAINING PROVIDED BY SUPPORT 

TEACHERS 



SAY: This section asks about the types of support that Support Teachers 
(Resource Teachers, Coordinators) in this school provide to other 
teachers. It asks about two kinds of support: first, to teachers in this 
school, and second, about support to teachers in other schools. 
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Q44. Is there a Mathematics Resource Teacher/Math Coordinator in this school? Circle 
ONE only. 

(a) Yes 

(b) No. If No, skip to Q47 





School Type 


Total 


NHP 


Comp 


No 


1 1 .6% 


56.1% 


33.3% 


Yes 


88.4% 


43.9% 


66.7% 


Total 


100.0% 


100.0% 


100.0% 



Q45. With which specific topics in mathematics has this teacher helped other teachers in 
this school? Please look at this list and tell me all the topics that the Math support teacher 
has helped other teachers with. CHECK ALL topics that respondents mention 

Making Mathematics Fun 

Geometry 

Statistics/Probability 

Measurement 

Numbers and Algebra 

Other 





NHP 


NHP 


Comp 


Yes 


Yes 


Math Fun 


51 


17 


Geometry 


48 


20 


Statistics 


34 


18 


Measurement 


47 


22 


Numbers & Algrbra 


37 


15 


Other 


10 


2 



Q46. Has this teacher provided any workshop for teachers in other schools? Circle 
ONE only. 

(a) Yes. Please describe the most recent workshop 

(b) No 





School Type 


Total 


NHP 


Comp 


No 


69.6% 


90.9% 


80.0% 


Yes 


30.4% 


9.1% 


20.0% 


Total 


100.0% 


100.0% 


100.0% 
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Q47. Is there a Literacy Resource Teacher/Literacy Coordinator in this school? Circle 
ONE only. 

(a) Yes 

(b) No. If No, skip to Q50 





School Type 


Total 


NHP 


Comp 


No 


8.7% 


53.0% 


30.4% 


Yes 


91 .3% 


47.0% 


69.6% 


Total 


100.0% 


100.0% 


100.0% 



Q48. With which specific topics in literacy has this teacher helped other teachers in this 
school? Please look at this list and tell me all the topics that the Literacy support teacher 
has helped other teachers with. CHECK ALL topics that respondents mention 

Emergent Literacy 

Use of Diagnostic Teaching of Reading 

Whole Language 

Phonemic Awareness 

Miscue Analysis 

Language Experience Approach 

Reading Writing Connection 

Writing Assessment 

Portfolios and Journals 

Eiteracy Window 

Other 





NHP 


NHP 


Comp 


Yes 


Yes 


Emergent 


23 


8 


Diagnostic 


42 


16 


Language 


33 


14 


Phonemic 


42 


21 


Miscue 


21 


10 


Language exp 


46 


21 


Reading 


52 


20 


Writing 


45 


19 


Portfolios 


57 


26 


Literacy 


50 


15 


other 


5 


1 
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Q49. Has this teacher provided any workshop for teachers in other schools? Circle 
ONE only. 

(a) Yes. Please describe the most recent workshop 

(b) No 





School Type 


Total 


NHP 


Comp 


No 


55.1% 


92.4% 


73.3% 


Yes 


44.9% 


7.6% 


26.7% 


Total 


100.0% 


100.0% 


100.0% 



Q50. Is there an Assessment Coordinator in this school? Circle ONE only. 

(a) Yes 

(b) No. If No, Skip to Q 52 





School Type 


Total 


NHP 


Comp 


No 


17.4% 


13.6% 


15.6% 


Yes 


82.6% 


86.4% 


84.4% 


Total 


100.0% 


100.0% 


100.0% 



Q51. Has this teacher provided any inservice training on assessment for other teachers in 
this school? Circle ONE only. 

(a) Yes 

(b) No 





School Type 


Total 


NHP 


Comp 


No 


30.4% 


25.8% 


28.1% 


Yes 


69.6% 


74.2% 


71 .9% 


Total 


100.0% 


100.0% 


100.0% 
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Q52. How do teachers in this school help each other improve classroom skills? CHECK 
ALL that respondents mention but DO NOT READ list 

Observing other teachers in this school (Intra school classroom observation) 

Observing teachers in another school (Inter- school classroom observation) 

Sharing of best practices through workshops 

Common planning time 

Other: Please describe 



Number of 
responses 


School Type 


Total 


NHP 


Comp 


1 


23.2% 


27.3% 


25.2% 


2 


34.8% 


47.0% 


40.7% 


3 


31 .9% 


18.2% 


25.2% 


4 


8.7% 


7.6% 


8.1% 


5 


1 .4% 




.7% 


Total 


100.0% 


100.0% 


100.0% 
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SECTION IX. COMMUNITY AND PARENT TRAINING 



SAY: This section asks about training that has been provided to parents of 
children in this school and to other community members who are 
associated with this school. 



Q53. What types of training have been offered to parents and other community members 
associated with this school, over the past 5 years? CHECK ALL that respondents 
mention but DO NOT READ list 

Training in school governance and leadership 

Parents trained as “parent training coordinators 

Parents trained in school governance and leadership 

Parents attending the National Parenting Conferences 

Parents participating in Parent Participation Facilitation Skills workshop 

Other: Please describe 



Number of 
responses 


School Type 


Total 


NHP 


Comp 


0 


2.9% 


9.1% 


5.9% 


1 


44.9% 


77.3% 


60.7% 


2 


26.1% 


10.6% 


18.5% 


3 


21 .7% 


3.0% 


12.6% 


4 


4.3% 




2.2% 


Total 


100.0% 


100.0% 


100.0% 



Q54. Has anyone been recruited from the local community and trained to sensitize and 
mobilize the community to support the school ? This could be a Community Participation 
Facilitator, Parent Representative or Parent Grade Coordinator. Circle ONE only. 

(a) Yes 

(b) Yes, but not specially trained 

(c) No 





School Type 


Total 


NHP 


Comp 


Missing 


6.9% 


1 .5% 


4.3% 


No 


12.5% 


46.3% 


28.8% 


YES 


61.1% 


19.4% 


41 .0% 


Yes but not trained 


19.4% 


32.8% 


25.9% 


Total 


100.0% 


100.0% 


100.0% 
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Q55. Has anyone from the local community (not a school official) made home visits to 
help develop relationships between parents and the school? Circle ONE only. 

(a) Yes 

(b) No 





School Type 


Total 


NHP 


Comp 


No 


52.2% 


65.2% 


58.5% 


Yes 


47.8% 


34.8% 


41 .5% 


Total 


100.0% 


100.0% 


100.0% 



Q56. How has the local community supported children’s learning in this school, over the 
past 5 years? Circle ONE only. 

(a) Some type of competition, such as a math competition, art competition, public 
speaking competition 

(b) Other 

(c) No support has been provided by local community 





School Type 


Total 


NHP 


Comp 


Missing 


4.2% 


3.0% 


3.6% 


No support 


16.7% 


31.3% 


23.7% 


Other 


65.3% 


46.3% 


56.1% 


Some 


13.9% 


1 9.4% 


1 6.5% 


Total 


100.0% 


100.0% 


100.0% 
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Q57. What sort of training has the School Board received? CHECK ALL that 
respondents mention but DO NOT READ list 

Workshops from the MOEYC 

Workshops from the National Council for Education 

Other 

None 





School Type 


Total 


NHP 


Comp 


MOEYC Workshops No 


59.4% 


51 .5% 


55.6% 


Yes 


40.6% 


48.5% 


44.4% 


Total 


100.0% 


100.0% 


100.0% 



Workshops from 
NCE 


School Type 




NHP 


Comp 


Total 


NCE No 


52.2% 


74.2% 


63.0% 


Yes 


47.8% 


25.8% 


37.0% 


Total 


100.0% 


100.0% 


100.0% 



Other 


School Type 


Total 




NHP 


Comp 




No 


78.3% 


87.9% 


83.0% 


Yes 


21 .7% 


12.1% 


17.0% 


Total 


100.0% 


100.0% 


100.0% 



None 


School Type 


Total 


NHP 


Comp 


No 


81.2% 


74.2% 


77.8% 


Yes 


1 8.8% 


25.8% 


22.2% 


Total 


100.0% 


100.0% 


100.0% 
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SECTION X. HEALTH AND NUTRITION 



Q58. During the past 5 years, what method has been used to identify “special needs” 
students in this school? CHECK ALL that respondents mention but DO NOT 
READ list 

Grade 1 students screened at National Health Fair 

Child’s doctor filled out a medical form 

Students screened professionally by psychologist/nurse 

Classroom teachers observed and identified students 

Don’t know 

Other 



Number of 
responses 


School Type 


Total 


NHP 


Comp 


1 


39.1% 


50.0% 


44.4% 


2 


55.1% 


45.5% 


50.4% 


3 


5.8% 


4.5% 


5.2% 


Total 


100.0% 


100.0% 


100.0% 



Q59. What sorts of programs have special needs students participated in? 





School Type 


Total 




NHP 


Comp 




Breakfast & lunch programme 


22.2% 


17.9% 


20.1% 


Breakfast programme 


12.5% 


3.0% 


7.9% 


Feeding programme 


15.3% 


23.9% 


19.4% 


Instructional programme 


6.9% 


3.0% 


5.0% 


Lunch programme 


13.9% 


14.9% 


14.4% 


Missing 


6.9% 


7.5% 


7.2% 


None 


8.3% 


10.4% 


9.4% 


Other 




4.5% 


2.2% 


PATH programme 


13.9% 


14.9% 


14.4% 


Total 


100.0% 


100.0% 


100.0% 



Q60. What types of school meals were available in this school last year (2004/2005)? 

CHECK ALL that respondents mention but DO NOT READ list 

Nutribun and drink 

Cooked meal (breakfast or lunch) 

Tuck shop snacks (e.g. patties) 

Other 
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Nutribun & Drink 


School Type 


Total 


NHP 


Comp 


No 


43.5% 


47.0% 


45.2% 


Yes 


56.5% 


53.0% 


54.8% 


Total 


1 00.0% 


100.0% 


100.0% 



Cooked Meal 


School Type 


Total 


NHP 


Comp 


No 


5.8% 


6.1% 


5.9% 


Yes 


94.2% 


93.9% 


94.1% 


Total 


100.0% 


100.0% 


100.0% 



Tuck Shop 


School Type 


Total 


NHP 


Comp 


No 


65.2% 


69.7% 


67.4% 


Yes 


34.8% 


30.3% 


32.6% 


Total 


100.0% 


100.0% 


100.0% 



Other 


School Type 


Total 


NHP 


Comp 


No 


94.2% 


90.9% 


92.6% 


Yes 


5.8% 


9.1% 


7.4% 


Total 


100.0% 


100.0% 


100.0% 
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Q61. On a daily basis, how regularly were meals served last year (2004/2005)? Please 
choose one option only. Circle ONE only. 

(a) Every day or nearly every day 

(b) About half the time (2-3 days per week) 

(c) Periodically, depending on resource availability 





Schooi Type 




NHP 


Comp 


Totai 


Missing 


5.6% 


3.0% 


4.3% 


35 




1 .5% 


.7% 


Every Day 


90.3% 


83.6% 


87.1% 


Haif the Time 




7.5% 


3.6% 


Periodicaiiy 


4.2% 


4.5% 


4.3% 


Totai 


100.0% 


100.0% 


100.0% 



Q62. Does this school provide a free meal (either breakfast or lunch) for any students? 

Circle ONE only. 

(a) Yes 

(b) No. If No, skip to Section XI 





Schooi Type 


Totai 


NHP 


Comp 


No 


2.9% 


3.0% 


3.0% 


Yes 


97.1% 


97.0% 


97.0% 


Totai 


100.0% 


100.0% 


100.0% 
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Q63. On average, how many children qualified for a free school meal (breakfast or 
lunch) each day during the last school year (2004/2005)? 





School Type 


Total 




NHP 


Comp 




1-10 


21 .2% 


41 .5% 


31.3% 


11-20 


24.2% 


27.7% 


26.0% 


21-30 


22.7% 


9.2% 


1 6.0% 


31-40 


6.1% 


4.6% 


5.3% 


41-50 


12.1% 


1 .5% 


6.9% 


51-60 


3.0% 


3.1% 


3.1% 


61-70 




3.1% 


1 .5% 


71-80 


3.0% 


1 .5% 


2.3% 


91-100 


3.0% 


4.6% 


3.8% 


> 100 


4.5% 


3.1% 


3.8% 


Total 


1 00.0% 


1 00.0% 


100.0% 



Q64. Has anyone monitored the growth (weight for height, BMI) of any children 
participating in a free school meal program? Circle ONE only. 

(a) Yes, someone in the school (such as a school nurse) 

(b) Yes, someone from the Ministry of Health 

(c) No 





School Type 






NHP 


Comp 


Total 


Missing 


6.9% 


3.0% 


5.0% 


No 


63.9% 


83.6% 


73.4% 


Yes In School 


15.3% 


1 1 .9% 


1 3.7% 


Yes Ministry of Health 


12.5% 


1 .5% 


7.2% 


Yes New Horizon 
Project's Nutritionist 


1 .4% 




.7% 


Total 


100.0% 


1 00.0% 


1 00.0% 
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SECTION XI. COMPUTER USE IN THE SCHOOL 



SAY: These questions ask about computers in this school. Is there 
someone else (such as a computer or Information Technology (IT) 
coordinator) who would be better able to answer these questions? If so, 
could this person be invited to join the discussion now? 



Q65. Does this school have computers for student use? Circle ONE only. 

(a) Yes, and all or most of them are working 

(b) Yes, but they are down now 

(c) No 





Schooi Type 






NHP 


Comp 


Totai 


Missing 


4.2% 


1 .5% 


2.9% 


No 


4.2% 


50.7% 


26.6% 


Yes But Down 


18.1% 


19.4% 


18.7% 


yes but oniy 
one computer 


1 .4% 




.7% 


Yes Working 


72.2% 


28.4% 


51.1% 


Totai 


100.0% 


100.0% 


100.0% 



Q66. Does this school have internet access? Circle ONE only. 

(a) Yes 

(b) Yes, but it is down now 

(c) No 





Schooi Type 




NHP 


Comp 


Totai 


Missing 


4.2% 


1 .5% 


2.9% 


No 


47.2% 


80.6% 


63.3% 


Yes But Down 


9.7% 


6.0% 


7.9% 


Yes 


38.9% 


1 1 .9% 


25.9% 


Totai 


100.0% 


100.0% 


100.0% 
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Q67. Is there a computer Resource Teacher in this school? Circle ONE only. 

(a) Yes 

(b) Yes, there should be, but the position is vacant right now 

(c) No. If No, Skip to Q69 





Schooi Type 


Totai 


NHP 


Comp 


Missing 


5.6% 


1 .5% 


3.6% 


No 


1 9.4% 


61.2% 


39.6% 


Yes 


65.3% 


35.8% 


51.1% 


Yes But Vacant 


9.7% 


1 .5% 


5.8% 


Totai 


100.0% 


100.0% 


100.0% 



Q68. What sort of training does the computer Resource Teacher provide for other 
teachers? Circle ONE only. 

(a) Workshops on how to use computers to teach 

(b) Other. 

(c) No training provided by computer Resource Teacher 





Schooi Type 


Totai 


NHP 


Comp 


Missing 


27.8% 


62.7% 


44.6% 


No Training 


8.3% 


1 1 .9% 


10.1% 


Other 


11.1% 


1 1 .9% 


1 1 .5% 


Workshops 


52.8% 


13.4% 


33.8% 


Totai 


1 00.0% 


1 00.0% 


1 00.0% 



Q69. Is there a functioning computer available for school records and other 
administrative purposes? Circle ONE only. 

(a) Yes 

(b) Not functioning now, but it was working in school year 2004-5 

(c) No 





Schooi Type 


Totai 


NHP 


Comp 


Missing 


5.6% 


1 .5% 


3.6% 


No 


2.8% 


1 7.9% 


10.1% 


Not Functioning now 


9.7% 


1 3.4% 


1 1 .5% 


Yes 


81.9% 


65.7% 


74.1% 


Yes, not for schooi record 




1 .5% 


.7% 


Totai 


1 00.0% 


1 00.0% 


1 00.0% 
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Q70. Has the Jamaica School Administrative System (JSAS) software been installed in a 
computer at this school, and if so, when? Circle ONE only. 

(a) Yes. If Yes, Ask “when was it installed” 

(b) No. If No Skip to Section XII 





School Type 


Total 


NHP 


Comp 


No 


7.2% 


90.9% 


48.1% 


Yes 


92.8% 


9.1% 


51 .9% 


Total 


100.0% 


100.0% 


100.0% 



Q71. Is there a JSAS support guide (user’s guide, manual) in this school? Circle ONE 
only. 



(a) Yes. If “yes”, ask to see to see the guide 

(b) No 





School Type 


Total 


NHP 


Comp 


No 


29.0% 


97.0% 


62.2% 


Yes 


71 .0% 


3.0% 


37.8% 


Total 


100.0% 


100.0% 


100.0% 
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Q72. Who in this school has received JSAS training? CHECK ALL respondents 
mention but DO NOT READ list 

Principal 

Specialist teacher 

Other specialist 

Other: Please describe 

No one has received JSAS training 



Principal 


School Type 




NHP 


Comp 


Total 


No 


66.7% 


93.9% 


80.0% 


Yes 


33.3% 


6.1% 


20.0% 


Total 


100.0% 


100.0% 


100.0% 



Specialist Teacher 


School Type 


Total 


NHP 


Comp 


No 


62.3% 


100.0% 


80.7% 


Yes 


37.7% 




1 9.3% 


Total 


1 00.0% 


100.0% 


100.0% 



Other Specialist 


School Type 


Total 


NHP 


Comp 


No 


88.4% 


100.0% 


94.1% 


Yes 


1 1 .6% 




5.9% 


Total 


100.0% 


100.0% 


100.0% 



Other 


School Type 


Total 


NHP 


Comp 


No 


40.6% 


97.0% 


68.1% 


Yes 


59.4% 


3.0% 


31.9% 


Total 


100.0% 


100.0% 


100.0% 



No one 


School Type 


Total 


NHP 


Comp 


No 


89.9% 


97.0% 


93.3% 


Yes 


10.1% 


3.0% 


6.7% 


Total 


100.0% 


100.0% 


100.0% 
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SECTION XII GRADE 4 LITERACY 



SAY: This last section is specifically about assessment, Grade 4 literacy 
and the Literacy Summer Camps offered for students after Grade 4. 



Q73. How many children were enrolled in Grade 4 last year (2004-5)? 
children 





NHP 


NHP 


Comp 


Mean 


Mean 




68 


53 



Q74. How many Grade 4 children in this school were recommended for Literacy Summer 
Camp last year? 

children 





NHP 


NHP 


Comp 


Mean 


Mean 




25 


22 



Q75. How many of these children actually attended a Literacy Summer Camp (either a 
day camp or a residential camp) in 2005? 

children 





NHP 


NHP 


Comp 


Mean 


Mean 




16 


13 
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Q76. In what years have Grade 4 students from this school attended a Residential 
Literacy Summer Camp? CHECK ALL that respondents mention 

Summer 2004 

Summer 2005 

No Grade 4 students have attended a Residential Literacy Summer Camp 





School Type 


Total 


NHP 


Comp 


Missing 


4.2% 


1 .5% 


2.9% 


No Grade 4 


68.1% 


67.2% 


67.6% 


Summer 2004 


6.9% 


9.0% 


7.9% 


Summer 2005 


20.8% 


22.4% 


21 .6% 


Total 


100.0% 


100.0% 


100.0% 



Q77. How many children took the Grade 4 literacy posttest after Literacy Summer 
Camp? 

children 





NHP 


NHP 


Comp 


Mean 


Mean 




16 


12 



Q78. How many children will repeat Grade 4 this school year (2005-6)? 
children 





NHP 


NHP 


Comp 


Mean 


Mean 




4 


3 
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Q79. How have teachers used the tests and results of the Grade 4 Literacy test in planning 
to teach? CHECK ALL that respondents mention 

to group students by ability within classrooms 

to help with streaming in 5'*^ grade 

Other: 





Schooi Type 


Totai 


NHP 


Comp 


Missing 


5.6% 


3.0% 


4.3% 


Other 


25.0% 


20.9% 


23.0% 


To Group students 


62.5% 


65.7% 


64.0% 


To heip with 5th Grade 


6.9% 


10.4% 


8.6% 


Totai 


1 00.0% 


1 00.0% 


1 00.0% 



Q80. How have Grade 4 teachers used the tests and results of the Grade 3 Diagnostic test 
in planning to teach? 







School Type 


Total 




NHP 


Comp 




Assess Strength & Weakness 


62.5% 


53.7% 


58.3% 




Did not use the test 


2.8% 


6.0% 


4.3% 




Group for instruction 


26.4% 


31.3% 


28.8% 




Missing 


4.2% 


1.5% 


2.9% 




Not used 


2.8% 


6.0% 


4.3% 




Used to adjust teaching 


1.4% 


1.5% 


1.4% 


Total 




100.0% 


100.0% 


100.0% 



Q81. Do 4'*^ Grade teachers receive their students’ individual scores from Grade 3 tests? 

Circle ONE only. 

(a) Yes 

(b) No 





Schooi Type 


Totai 


NHP 


Comp 


No 


17.4% 


19.7% 


18.5% 


Yes 


82.6% 


80.3% 


81 .5% 


Totai 


100.0% 


100.0% 


100.0% 
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Q82. How have teachers used the tests and results of the Grade 1 Readiness Assessment 
in planning to teach? 



School Type 

NHP Comp 



number Percent number Percent 



Assess Strength and Weakness 


42 


58.3% 


29 


43.3% 


Do not use data 


1 


1.4% 


1 


1.5% 


Group for Instruction 


15 


20.8% 


23 


34.3% 


Missing 


5 


6.9% 


1 


1.5% 


Used to adjust teaching 


9 


12.5% 


13 


19.4% 
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Question 83 Do you have a copy of the 2005 Grade 4 Literacy Test Class Report form(s) 
for this school? 

A. Yes 

B. No 





School Type 


Total 


NHP 


Comp 


No 


10.1% 


7.6% 


8.9% 


Yes 


89.9% 


92.4% 


91.1% 


Total 


100.0% 


100.0% 


100.0% 



SAY: Do you have a copy of the results of the 2004/5 Grade 4 Literacy test, 
that is, a copy of the form that was sent to the MOEYC? If so, may I please 
see it, so that I can complete the attached form. COPY NUMBERS FROM 
THE FORM INTO THE ATTACHED BLANK FORM. 



SAY: Thank you very much. We will send you a summary of the results 
from this survey when it is finished. 
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